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Abstract 

The  purpose  of  this  thesis  was  to  determine  if  current 
supply  management  indicators  provide  supply  managers  with 
c-uality  measurement  tools.  This  research  developed  the 
Quality-Based  Metrics  Framework,  consisting  of  customer 
focus  and  continuous  process  improvement  with  underlying 
supporting  elements  of  time  and  cost. 

A  literature  review  determined  an  appropriate  framework 
for  improving  logistics  indicators.  Interviews  with  supply 
analysts  and  a  review  of  pertinent  publications  identified 
the  most  commonly  used  supply  management  indicators.  A 
focus  group  of  highly  experienced  senior  supply  officers 
determined  the  usefulness  of  currently  used  indicators  in 
terms  of  the  framework.  Using  the  input  from  the  focus 
group,  the  authors  developed  a  revised  list  of  metrics  which 
strongly  reflected  the  elements  of  the  framework. 

The  revised  metrics  were  evaluated  by  the  same  focus 
group  and  found  to  be  more  useful  in  terms  of  the  framework. 
These  results  were  validated  by  surveying  a  sample  of  senior 
supply  managers  throughout  the  Air  Force.  The  results 
indicate  that  the  current  supply  indicators  could  be 
improved  to  provide  managers  with  better  quality  management 
tools . 


viii 


QUALITY  BASED  SUPPLY  MANAGEMENT  INDICATORS 


I .  Introduction 


Background 

General .  The  United  States,  along  with  the  rest  of  the 
world,  has  seen  tremendous  change  in  the  last  decade.  This 
change  includes  the  reunification  of  Germany,  the  appearance 
of  new  countries  as  a  result  of  the  break-up  of  the  USSR, 
and  the  end  of  the  Cold  War.  Americans  are  faced  with  a 
different  world  today,  and  the  uncertainty  that  follows  such 
dramatic  change. 

As  a  result  of  the  changing  world  and  the  end  of  the 
Cold  War  Era,  the  perceived  need  for  a  large  military  has 
dramatically  decreased.  Now  that  the  military  is  no  longer 
viewed  as  such  a  vital  component  to  the  national  interest  as 
compared  to  the  economy,  the  American  people  have  asked  for 
the  "peace  dividend."  This  peace  dividend  equates  to  taking 
the  money  that  was  previously  spent  on  the  military  and 
funneling  it  to  domestic  and  social  issues  which  are 
perceived  as  being  in  more  desperate  need  of  funds. 

The  population  of  military  personnel  has  decreased  from 
a  peak  of  2,174,000  in  1987  to  1,886,000  in  1992,  reflecting 
the  changing  world  and  satisfying  the  wishes  of  Americans  to 
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spend  less  on  defense  (AF  Magazine,  1993:48).  It  is 
projected  to  decrease  even  further. 

Along  with  a  reduction  in  personnel,  there  is  an 
ongoing  reduction  in  the  total  number  of  bases  and  weapon 
systems.  For  the  United  States  Air  Force  (USAF) ,  the 
obvious  reduction  is  in  the  number  of  bases  and  the  total 
number  of  aircraft.  Since  fiscal  year  1989,  the  number  of 
major  installations  worldwide  has  dropped  from  141  to  124. 
The  total  number  of  aircraft  in  the  active  duty  Air  Force 
inventory  was  6724  as  of  30  September  1990,  from  a  previous 
peak  of  7299  as  of  September  1985  (AF  Magazine,  1992:24) . 
These  dramatic  cuts  require  an  efficient  logistics  system  in 
order  to  maintain  national  defense. 

Standard  Base  Supply  System.  Base  supply  plays  an 

important  role  in  the  overall  logistics  functions  of  the  Air 

Force,  as  can  be  seen  in  the  following  passages: 

Because  ...  a  ready  supply  of  dependable  spare  parts 
are  essential  to  a  strong  air  defense  . . .  reliable 
products  and  prompt  services  is  today  an  inseparable 
component  of  modern  Air  Force  logistics. 

(Brownlee,  1989:4) 

Aircraft  grounded  for  lack  of  parts  or  consumables 
represent,  for  the  period  they  are  out  of  service,  as 
much  loss  to  combat  capability  as  aircraft  destroyed  by 
enemy  action.  (DAF,  March  1992:255) 

The  USAF  uses  what  is  known  as  the  Standard  Base  Supply 
System  (SBSS)  to  support  the  maintenance  and  operations  of 
all  aircraft  at  the  wing  level.  The  SBSS  is  defined  as  an 
accounting  system  consisting  of  standardized  computer 
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equipment,  programs,  procedures,  and  supply  policy  to 
provide  base  activities  with  their  supply  needs  and  accounts 
for  supplies,  equipment,  fuel,  munitions,  and  clothing  (DAF, 
July  1992 :Ch  1,  7) . 

A  Chief  of  Supply  (COS)  is  charged  with  operating  the 
SBSS.  The  COS  is  responsible  to  the  wing  commander  for  an 
effective  and  efficient  base  supply  operation.  A  typical 
supply  squadron  consists  of  approximately  200  men  and  women, 
comprised  of  both  military  and  civilian  personnel.  The 
squadron  is  normally  broken  down  to  five  smaller  sections, 
referred  to  as  flights,  which  are  managed  by  supply  officers 
(flight  chief)  and  senior  NCOs  (flight  superintendent) . 
Within  each  flight  are  subsections,  referred  to  as  sections 
(see  Figure  1) .  The  Chief  of  Supply  relies  on  the  flight 
chiefs  and  superintendents  to  make  daily  decisions  which 
affect  the  supply  account. 

The  Supply  Management  Report  (M-32)  is  the  most 
comprehensive  management  report  available  to  supply  managers 
and  the  primary  source  of  management  indicators  used  by 
base-level  supply  managers  (DAF,  July  1992:Ch  5,  395).  The 
M-32  is  published  at  the  end  of  every  month  in  a  standard 
format  and  shows  the  aggregated  monthly  status  for  various 
supply  functions . 


CHIEF  OF  SUPPLY  ORGANIZATION  CHART 

Figure  1.  Base  Supply  Organizational  Chart 


Total  Quality  Management.  Total  Quality  Management 


(TQM)  is  the  generic  term  that  was  adopted  by  American 
industry  and  government  for  quality  management  programs . 

TQM  evolved  in  America  but  became  famous  through  effective 
implementation  in  Japan  (Krone,  1990:35).  There  is  no 
simple  definition  of  Total  Quality  Management  (TQM) ,  and 
very  few  people  fully  understand  the  philosophy  well  enough 
to  implement  it.  "Because  of  the  lack  of  full  understanding 
of  what  TQM  is  and  the  absence  of  clearly  specified 
objectives,  success  has  been  sporadic  and  progress  across 
large  organizations  very  slow"  (Emmelhainz,  1991:35). 
According  to  Deming,  a  leader  in  TQM,  there  are  14 
obligations  of  top  management  to  eliminate  obstacles  that 
hinder  quality  improvements,  many  of  which  were  established 
by  management .  Following  are  some  of  the  management 
obligations  that  apply  to  continuous  process  improvement: 

-  Constantly  improve  production 

-  Create  constancy  of  purpose  toward  improvement 

-  Replace  quotas  and  eliminate  management  by  objective 

-  Eliminate  slogans,  exhortations,  and  targets 

-  Cease  dependence  on  inspection  to  achieve  quality 

-  Institute  leadership  (Deming,  1986:23-24) 

According  to  a  DOD  policy,  the  hallmark  of  the  TQM 
process  is  continuous  improvement  (DOD,  1990:2) .  Quality 
improvement  and  process  measurement  must  have  objectives. 
First,  the  customers  have  to  be  identified  and  their 
requirements  made  known  in  order  to  consistently  meet  their 
needs  and  expectations  (DOD,  1990:4).  According  to  Saylor: 
"Customer  is  everyone  affected  by  the  product  or  service" 
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(Saylor,  1990:20).  Once  the  customer  is  identified, 

performance  measures  need  to  be  established  to  support  the 

concept  of  continuous  improvement.  The  Federal  Total 

Quality  Management  Handbook  describes  quality  this  way: 

Some  people  are  confused  by  the  term  quality,  because 
it  has  many  different  meanings  depending  on  the 
context.  When  it  is  referred  to  as  part  of  the  TQM 
effort  in  the  federal  government,  quality  is  defined  as 
"meeting  the  customer's  requirements,  needs,  and 
expectations,  the  first  time  and  every  time."  Who 
determines  what  quality  is?  Our  customers,  inside  and 
outside  each  organization.  Congress,  which  represents 
the  public  is  included.  Customer  satisfaction  is 
paramount.  (Hyde,  1990-91:16) 

The  message  behind  this  definition  is  that  any  concept  of 

quality  must  be  customer  focused. 


TQM  in  Base  Supply.  The  concept  of  continuous 
improvement  is  especially  salient  in  supply  management  due 
to  the  current  supply  measurement  system.  While  the  primary 
emphasis  is  placed  on  supporting  the  operational  mission 
through  stocking  and  requisitioning  aircraft  spare  parts, 
supply  managers  are  measured  against  a  different  set  of 
standards  during  the  monthly  analysis  program  called  the 
Chief  of  Supply  "How  Goes  It"  (Greer  and  Moon,  1981:32). 
Because  the  main  customer  of  base  supply  is  the  maintenance 
organization,  the  standards  should  be  focused  on  how  well 
they  are  supporting  the  main  customer. 

During  the  "How  Goes  It"  session,  current  indicators 
are  compared  against  Major  Command  (MAJCOM)  imposed  targets. 
The  potential  danger  with  this  method  of  measurement  is  that 
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if  the  various  functions  are  meeting  or  exceeding  the  stated 
goal,  no  further  research  is  required.  Only  when  a  stated 
goal  is  not  met  is  research  conducted.  This  method  seems  to 
go  against  three  of  Deming's  obligations  of  top  management: 
Eliminate  management  by  objective;  eliminate  targets;  and 
inspecting  after  the  fact,  rather  than  continuously  striving 
to  improve . 

Another  obligation  of  top  management  is  that  managers 
must  take  an  active  leadership  role  in  improving  the  quality 
within  the  organization.  Despite  the  fact  that  there  is  a 
tool  to  keep  track  of  the  supply  account  on  a  daily  basis, 
due  to  the  sheer  volume  of  information,  most  managers  are 
content  to  wait  until  the  M-32  is  published  at  the  end  of 
the  month  (LeSage,  1989:8).  By  waiting  until  the  end  of  the 
month,  supply  managers  are  taking  a  reactive  rather  than  a 
proactive  leadership  role . 

The  format  of  the  M-32  has  remained  unchanged  since  the 
Air  Force  began  implementing  TQM  and  may  not  meet  the 
current  criteria  for  an  effective  measurement  as  outlined  in 
Total  Quality  Management  philosophy.  Incorporating  base 
supply  management  reports,  specifically  the  M-32,  into  the 
Quality  Air  Force  (QAF)  philosophy  may  require  restructuring 
the  M-32  to  enhance  concept  of  continuous  process 
improvement  (DOD,  1990:iii). 
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Specific  Problem 


The  management  indicators  that  supply  managers  are 
using  may  not  be  consistent  with  the  TQM  philosophy. 

Objective  Statement 

The  objective  of  this  research  is  to  determine  which 
management  indicators  should  be  used  in  order  to  provide 
supply  managers  with  quality  measurement  tools. 

Investigative  Questions 

The  questions  that  must  be  answered  in  order  to  solve 
this  specific  problem  are: 

1 .  According  to  the  TQM  philosophy,  what  are  the 
appropriate  characteristics  which  management 
indicators  should  possess  in  order  to  assess  the 
performance  of  base  supply? 

2 .  What  are  the  most  commonly  used  supply  management 
indicators? 

3.  In  terms  of  TQM,  how  useful  are  these  indicators  to 
base  supply  officers? 

4.  Can  these  supply  indicators  be  modified  so  that 
they  are  more  in-line  with  the  TQM  philosophy? 

Scope 

The  scope  of  this  thesis  will  be  limited  to  base-level 
supply  accounts,  and  information  will  be  gathered  only  from 
supply  officers  and  managers  with  at  least  ten  years  of 
base-level  supply  experience.  The  emphasis  will  be  on  the 
M-32  (Supply  Management  Report)  and  the  role  it  will  play  in 
the  new  Air  Force  style  of  a  Quality  Air  Force.  This  thesis 
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will  not  attempt  to  cover  indicators  from  a  systems 
perspective,  even  though  improvements  to  the  current  system 
might  possibly  be  made  by  linking  the  measurement  systems  of 
the  several  echelons . 

Justification 

This  research  is  required  for  three  reasons.  First, 
although  there  is  a  plethora  of  research  on  quality 
processes  in  manufacturing  organizations,  there  is  little 
information  on  what  specific  quality  measures  to  use  in 
logistics  areas.  Second,  the  dramatic  budget  cuts  over  the 
past  few  years  require  the  military  to  operate  more  like  a 
business  and  maximize  effectiveness  while  minimizing  costs. 
Finally,  the  Air  Force  is  implementing  quality  programs 
service-wide,  yet  logisticians  are  still  using  the  same 
management  indicators  that  they  have  used  for  many  years. 

Summary 

The  Air  Force  is  undergoing  major  reductions  in  the 
face  of  dramatic  global  political  changes.  The  emphasis  is 
being  placed  on  the  Air  Force  to  operate  more  effectively 
and  efficiently.  In  order  to  achieve  this  goal,  the  Air 
Force  has  adopted  the  TQM  philosophy.  The  SBSS  is  an 
integral  part  of  overall  Air  Force  operations.  However,  its 
measurement  system  has  remained  unchanged  since  the  adoption 
of  the  TQM  philosophy.  As  the  primary  management 
measurement  report  for  base  supply,  the  M-32  does  not  appear 
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to  be  providing  top  management  with  measures  consistent  with 
the  TQM  philosophy. 

Overview 

This  chapter  introduced  some  of  the  factors  which  are 
driving  the  need  to  improve  current  base  supply  management 
indicators.  Chapter  two  provides  an  in-depth  review  of  TQM 
in  the  Department  of  Defense  and  what  constitutes  a  good 
supply  indicator  in  terms  of  TQM  philosophy.  Chapter  three 
focuses  on  the  methodology  used  to  accomplish  the  research. 
The  fourth  chapter  presents  the  findings  and  results  of  the 
research.  Chapter  five  will  present  conclusions  and 
recommendations  in  the  area  of  continuous  improvement  for 
base  supply  management. 
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II .  Literature  Review 


Introduction 

Kev  Terms .  The  purpose  of  this  review  is  to  analyze 
current  literature  relating  to  quality  improvement  and  the 
management  indicators  used  in  measuring  quality  levels. 

This  review  will  determine  what  constitutes  a  good  measure, 
with  an  emphasis  on  continuous  process  improvement  and 
customer  focus  based  on  TQM  philosophy.  It  will  focus  on 
measures  within  the  field  of  logistics. 

The  terms  measures,  management  indicators,  and 
indicators  all  have  the  same  meaning  and  will  be  used 
interchangeably  throughout  this  thesis.  These  terms  should 
be  distinguished  from  the  term  metrics. 

Metrics  are  nothing  more  than  meaningful  measures. 
For  a  measure  to  be  meaningful,  however,  it  must 
present  data  that  allow  us  to  take  action.  It  must  be 
customer  oriented  and  support  the  meeting  of  our 
organizational  goals  and  objectives.  Metrics  foster 
process  understanding  and  motivate  action  to 
continually  improve  the  way  we  do  business.  This  is 
distinguished  from  measurement,  in  that,  measurement 
does  not  necessarily  result  in  process  improvement. 
Good  metrics  always  will.  (AFSC,  1991:1-1) 


Quality  improvement  is  known  by  many  titles.  Quality 
Control,  Total  Quality  Improvement,  Total  Quality  Control, 
and  Total  Quality  Management  are  all  familiar  terms  for  the 
same  fundamental  philosophy.  This  philosophy,  as  it  applies 
to  the  Department  of  Defense,  is  derived  mainly  from  the 
writings  of  Dr.  W.  Edwards  Deming  and  Dr.  J.  M.  Juran  (DOD, 
1988:1).  Total  Quality  Management  (TQM)  is  defined  as: 
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. . .  both  a  philosophy  and  a  set  of  guiding  principles 
that  represent  the  foundation  of  a  continuously 
improving  organization.  TQM  is  the  application  of 
quanti*  jtive  methods  and  human  resources  to  improve  the 
material  and  services  supplied  to  an  organization,  all 
the  processes  within  an  organization,  and  the  degree  to 
which  the  needs  of  the  customer  are  met,  now  and  in  the 
future,  [emphasis  added] (OASD,  1989:1) 

Based  upon  this  definition,  TQM  can  be  described  as 

continuously  enhancing  customer  service  by  improving 

internal  processes  through  quantitative  methods.  One  item 

that  TQM  should  address  specifically  is  what  measurement 

system  should  be  developed  to  support  the  quality  program 

(Hyde,  1990-91:17). 


Quality  Measurement.  The  importance  of  quality 

measurement  is  reflected  in  the  following  passage: 

You  cannot  manage  what  you  cannot  measure.  You  cannot 
measure  what  you  cannot  operationally  define.  You 
cannot  operationally  define  what  you  do  not 
understand. .. You  will  not  succeed  if  you  do  not  manage. 
(VPC,  1989:74) 

There  are  two  kinds  of  change  that  the  Air  Force  is 
currently  undergoing.  The  first  type  of  change  is  something 
that  we  have  no  control  over,  the  top-down  mandated 
reduction  in  the  size  of  the  military  due  to  the  public 
perception  of  increased  world  stability.  The  second  type  of 
change  is  internal  and  is  caused  by  our  desire  to  become  a 
Quality  Air  Force  (McPeak,  1992) .  These  two  changes  are 
providing  the  impetus  for  Air  Force  personnel  to  do  things 
smarter.  Because  of  these  changes,  the  Air  Force  needs  to 
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review  its  current  measurement  systems  to  make  sure  that 

measures  are  consistent  with  TQM  goals. 

The  Wang  corporation  recently  underwent  a  change,  less 

drastic  than  the  one  faced  by  the  Air  Force  today,  when 

they  introduced  just-in-time  manufacturing  methods.  This 

change  forced  them  to  alter  their  entire  performance 

measurement  system.  Wang  developed  a  new  measurement 

approach,  called  Strategic  Measurement  Analysis  &  Reporting 

Technique  (SMART).  SMART'S  goal  is  to  integrate  both 

financial  and  nonfinancial  reporting,  to  link  operational 

goals  to  strategic  goals,  and  to  concentrate  the  measurement 

system  design  on  satisfying  customer  needs  and  fostering 

constant  evolution  (Dixon  and  others,  1990:52) . 

Implementation  of  the  SMART  system  resulted  in  some 

indicators  being  deleted  and  some  being  modified.  The 

revised  indicators  now  meet  the  criteria  of  metrics. 

Approximately  40  percent  of  the  existing  measures 
were  discarded,  including  purchased  price 
variance,  labor  productivity,  and  virtually  all  of 
the  standard  cost  accounting  variances  for 
operating  feedback. 

Examples  of  new  metrics  are  new  measures  of 
inventory  turns,  plug  and  play  rates,  material 
availability,  waste  rates,  and  process  times. 

(Dixon  and  others,  1990:55) 

The  TQM  concepts  of  satisfying  customer  needs  and 
fostering  continuous  improvement  are  discussed  in  the  next 
section . 
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Framework 


The  framework  for  this  discussion  consists  of  customer 
focus  and  continuous  process  improvement  as  they  relate  to 
quality  based  indicators.  The  quality  measurement  section 
provided  a  brief  overview  of  why  Air  Force  supply  managers 
must  reevaluate  the  current  measurement  system  by  focusing 
on  customer  needs  and  fostering  continuous  improvement. 

The  literature  review  revealed  that  two  key  areas  of 
TQM  philosophy,  customer  focus  and  continuous  process 
improvement,  constitute  a  basic  guideline  for  a  quality 
measurement  system.  These  two  areas  coincide  with  those 
that  Wang  Corporation  focused  on  when  implementing  their  new 
measurement  system.  While  TQM  literature  emphasizes 
customer  focus  and  continuous  process  improvement,  business 
and  application-oriented  literature  focuses  more  on  tangible 
and  practical  issues  such  as  the  "bottom  line,"  which  can  be 
judged  and  measured  by  time  and  cost  factors.  Thus,  the  two 
key  areas  of  customer  focus  and  continuous  process 
improvement  are  supported  by  both  time  and  cost  factors.  An 
example  of  how  one  aspect  of  the  framework  can  be  analyzed 
by  time  and  cost  factors  is  shown  in  the  case  of  Northern 
Telecom  Inc.,  where  an  emphasis  in  reduction  of  production 
time  resulted  in  lower  inventory  and  overhead  cost  and  an 
improvement  in  quality  (Merrills,  1989:109). 

The  framework  described  above  will  henceforth  be 
referred  to  as  the  Quality-Based  Metrics  Framework. 
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Customer  Focus.  One  of  the  basic  tenets  of  TQM  is 


complete  customer  focus  (Emmelhainz,  1991:35;  Hyde, 
1990-91:16-17).  All  measures  and  processes  must  be  customer 
focused,  whether  the  customers  are  internal  or  external 
customers.  Research  shows  that,  within  logistics,  quality 
programs  are  most  fully  implemented  in  the  area  of  customer 
service  (Read  and  Miller,  1991:34).  Improved  customer 
service  is  a  major  goal  of  TQM.  It  is  therefore  logical  to 
place  initial  emphasis  on  customer  service.  By  doing  this, 
an  organization  can  show  immediate,  short-term  results  from 
its  TQM  efforts  through  improved  customer  satisfaction. 

In  logistics,  customer  service  level  is  expressed  as 
the  percentage  of  demand  filled  within  a  specified  time 
period  after  receipt  of  the  customer's  order  (Tersine, 
1988:211).  Achieving  the  required  customer  service  level 
requires  the  active  participation  of  all  facets  of  the 
organization  (Motiska  and  Shilliff,  1988:27).  Many 
logistics  functions,  such  as  transportation,  storage, 
receiving,  inventory  management,  forecasting,  purchasing, 
and  planning,  indirectly  contribute  to  the  overall  customer 
service  level.  Without  improvements  in  these  related  areas, 
customer  service  improvement  will  be  limited. 

A  question  related  to  the  strategic  focus  of  TQM  is 
whether  or  not  the  quality  performance  measures  used  by 
organizations  actually  reflect  customer  priorities.  For 
example,  the  percentage  of  on-time  shipments  received  by  the 
supplier  is  not  an  indicator  which  concerns  the  customer  of 
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the  supplier.  The  customer  only  measures  whether  or  not  his 
or  her  shipment  arrives  on  time  from  the  supplier.  What  is 
important  to  the  customer  is  what  logisticians  should 
measure  (Trunick  and  Richardson,  1990:19). 

Customer  focus  is  one  important  aspect  of  a  good 
indicator.  Customer  support  is  also  a  determinant  of  the 
effectiveness  of  a  supply  organization.  Therefore,  one  of 
the  characteristics  that  supply  management  indicators  should 
possess  is  customer  focus.  This  focus  should  be  directed 
towards  processes  to  improve  overall  service  and  aimed  at 
both  internal  and  external  customers. 

Continuous  Process  Improvement.  Another  basic  tenet  of 
TQM  is  continuous  process  improvement  (Emmelhainz,  1991:35- 
36) .  Process  improvement  can  be  described  as  breaking  down 
all  logistics  functions  into  well-defined  activities  and 
then  improving  lower  level  activities  that  link  to  the 
critical  logistics  goals.  (Beischel  and  Smith,  1991:25-26) 
"Performance  measurement  is  built  into  the  processes  of 
identifying  and  overcoming  problems  that  lie  at  the  heart  of 
continuous  improvement"  (Hall  and  others,  1991:v). 

Continuous  process  improvement  relies  on  building 
quality  into  a  product  or  service,  not  inspecting  it  in 
(Prowse,  1990:5).  This  entails  looking  in  detail  at  the 
steps  involved  in  a  process  rather  than  just  the  outcome  of 
the  procedures.  The  Federal  Quality  Institute  identified 
measurement  and  analysis  of  processes  as  key  factors  in 
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determining  successful  quality  efforts  (Hyde,  1990-91:17). 
Indicators  show  how  efficiently  an  organization  is  achieving 
its  goals  and  identify  potential  areas  for  improvement. 

Looking  at  continuous  process  improvement  strengthens 
the  question  as  to  whether  customer  service  is  the  best 
indicator  of  quality  improvement.  Customer  service  is  the 
product  of  all  logistics  processes.  The  logistics  processes 
are  the  means  to  achieving  the  desired  level  of  customer 
service . 

Measures  that  are  taken  at  the  end  of  processes  are 
primarily  of  use  to  quality  planners.  The  people 
responsible  for  improving  the  operations,  on  the  other  hand, 
need  measures  taken  during  operations  (Juran,  1988:97). 
Supply  management  indicators  should  be  process  oriented 
rather  than  post-operations  summaries  if  they  are  to  be  used 
to  improve  quality  of  the  supply  processes.  An  example  of  a 
process-oriented  measure  is  the  daily  reject  listing  which 
tracks  the  number  of  wrong  inputs  into  the  base  supply 
system  and  provides  immediate  feedback  to  the  operator. 

Supporting  Elements .  Both  time  and  cost  factors  are 
integral  supporting  elements  for  measuring  the  level  of 
customer  focus  and  continuous  process  improvement. 

Customers  naturally  want  their  product  or  service  as  soon  as 
possible  at  the  lowest  price.  In  the  past,  cost  was  not  a 
major  issue  within  the  DOD  setting.  However,  due  to  the 
top-down  changes  in  cost  accounting,  cost  factors  are  having 
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an  increasingly  greater  impact  on  Air  Force  operations  and 
should  be  considered  when  measuring  processes. 

On  the  surface,  the  distinction  between  time  and  cost 
is  obvious,  speed  versus  money.  However,  in  service 
organizations  such  as  base  supply,  this  distinction  can 
become  blurred.  An  example  of  this  would  be  backorder 
costs.  Backorder  costs  relate  to  cost  in  that  they  are  the 
dollar  values  associated  with  having  to  place  a  backorder 
for  an  item  that  is  needed  but  is  not  in  stock.  They  relate 
to  time  in  that  backorders  cause  customers  to  wait  for 
needed  assets.  Because  customers  associate  service  with  the 
amount  it  takes  to  receive  their  goods,  supply  often  must 
expedite  shipments  to  please  customers.  This  increases 
ordering  and  transportation  costs. 

Time .  In  a  society  where  instant  gratification  is 
taken  for  granted,  time  is  indeed  money.  Consumers  are 
generally  willing  to  spend  more  for  a  service  or  a  product 
if  they  perceive  the  quality  to  be  worth  the  price.  In  this 
setting,  customer  focus  equates  to  speed.  This  may  explain 
the  growing  popularity  of  overnight  delivery  services  such 
as  Federal  Express,  Airborne  Express,  and  Express  Mail. 
Customers  want  to  receive  their  service  or  product  as  soon 
as  possible.  "Customer  satisfaction  is  the  'bottom  line,' 
and  since  time  is  very  important  to  customers,  it  should  be 
[important]  to  us."  (Tyndall,  1990:9) 

Fast  must  be  defined  as  it  relates  to  the  customer. 
Service  is  not  fast  unless  the  customer  says  it  is  fast. 
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"Terms  such  as  lead  times,  on-time  deliveries,  cycle  times, 
and  response  times  represent  customer  service  needs." 
(Tyndall,  1990:9)  Coincidentally,  base  supply  tracks  these 
same  four  time— based  indicators.  Lead  times  are  tracked  as 
the  average  time  for  base  supply  to  receive  requisitions 
from  sources  of  supply.  On— time  deliveries  are  tracked  as  a 
ratio  of  on— time  deliveries  to  total  deliveries.  Response 
times  are  tracked  in  minutes  by  the  priority  of  the 
requisition.  However,  both  on— time  delivery  and  response 
time  only  track  the  time  from  the  base  supply  warehouse  to 
the  customer.  The  time  required  to  receive* the  assets  from 
sources  of  supply  is  not  taken  into  consideration.  Cycle 
time  is  tracked  for  reparable  assets  in  the  form  of  Repair 
Cycle  Time. 

Lead  time  can  be  equated  to  the  logistics  pipeline  from 
various  sources  of  supply  to  base  supply.  The  logistics 
pipeline  can  be  defined  as:  "An  encompassing  system  through 
which  materiel  or  personnel  flow  from  sources  of  procurement 
to  their  point  of  use."  (Compendium  AFIT,  1981:522)  A 
recent  study  showed  that  a  reduction  of  one  day  lead  time  in 
the  Air  Force  logistics  pipeline  can  equate  to  approximately 
$50  million  in  savings  (Bond  and  Ruth,  1989:2).  On-time 
delivery  and  the  percentage  of  on-time  delivery  are  highly 
visible  measures  that  base  supply  tracks  daily  and  monthly, 
respectively.  On-time  delivery  of  parts  to  flightline 
repair  shops  is  deemed  critical  to  supporting  the  flying 
mission. 
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The  cycle  time  equivalent  is  repair  cycle  time.  Base 
supply  keeps  track  of  the  turnaround  time  for  reparable 
items,,  providing  a  form  of  measure  for  the  maintenance 
organization  and  creating  an  incentive  to  continuously 
improve  the  time.  Response  time  can  be  measured  in  base 
supply  by  "days  to  process  requisitions."  Including  time 
factors  in  measures  has  strong  implications  for  customer 
service  and  continuous  improvement. 

Two  critical  questions  must  be  answered  with  regard  to 
measurement.  The  first  question  deals  with  when  the  data 
will  be  collected,  and  the  second  question  deals  with  when 
the  data  will  be  reported  (Kinlaw,  1992:72).  The  M-32  is  a 
report  that  is  produced  monthly  and  can  only  be  produced  on 
an  as-required  basis  with  a  great  amount  of  difficulty.  "We 
cannot  possibly  commit  to  continuous  improvement  if  we 
collect  data  only  when  we  have  a  problem  rather  than 
collecting  data  as  a  way  of  doing  business."  (Kinlaw, 

1992:16)  As  noted  before,  Juran  states  that  measures  must 
be  taken  during  a  process  to  be  of  use  to  those  responsible 
for  improving  the  process,  and  "real-time  information  is 
necessary  to  permit  prompt  detection  and  correction  of 
nonconformance  to  goals."  (Juran,  1989:156) 

Cost .  Like  most  government  agencies,  the  base 
supply  organization,  despite  its  function  as  a  retail  sales 
store  for  the  base,  does  not  operate  in  a  profit-seeking 
environment.  However,  as  a  result  of  Executive  Order  12637 
signed  by  President  Ronald  Reagan  in  1988,  which  established 
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a  government-wide  program  to  improve  the  quality, 
timeliness,  and  efficiency  of  the  federal  government, 
techniques  used  in  profit-seeking  organizations  have  been 
advocated  for  use  in  the  federal  government  (Harr  and 
Godfrey,  1992:52).  An  immediate  effect  for  the  Air  Force 
has  been  the  Defense  Management  Review  Decision  (DMRD)  901, 
which  calls  for  a  major  reduction  in  the  Air  Force 
inventory . 

The  Air  Force  Materiel  Command  (AFMC)  has  a  dedicated 
"Inventory  Reduction  Team"  whose  main  function  is  to  reduce 
inventory  throughout  the  Air  Force.  The  purpose  of  reducing 
the  inventory  is  to  lower  the  overhead  cost  associated  with 
carrying  excess  inventory.  Management  of  inventory  is  a 
continuous  process  (Illsley,  1992) . 

Number  of  outstanding  back  orders  is  a  direct 
reflection  of  customer  service.  Customers  want  the  product 
or  service  they  paid  for  immediately.  The  philosophy  for 
back  orders  in  base  supply  is  similar  to  a  profit  seeking 
organization  in  that  both  set  a  customer  service  level 
knowing  that  there  will  be  a  certain  number  of  backorders. 
The  customer  service  level  is  a  management  decision  taking 
into  consideration  ordering  costs  versus  holding  costs. 

With  the  implementation  of  DMRD  904,  the  initiative  to  allow 
local  commanders  to  have  more  control  over  how  their  funds 
are  spent,  backorders  and  their  associated  costs  will  be 
scrutinized  ever  more  closely. 
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Logistics  Quality  Indicators.  "The  biggest  stumbling- 

blocks  to  quality  now  are  the  lack  of  clarity  on  what  to 

measure,  and  the  lack  of  information  systems  to  support  such 

measurement"  (Read  and  Miller,  1991:36).  In  addition  to 

this  observation.  Read  and  Miller  found  that  past 

performance  measures  need  to  be  changed  in  order  to  align 

them  with  quality  objectives. 

Management  indicators  are  necessary  tools  in 

determining  the  success  of  logistics  organizations.  The  old 

adage  "use  the  proper  tool  for  the  proper  job"  is  as 

appropriate  here  as  it  is  elsewhere.  Management  indicators* 

that  are  not  in  sync  with  the  strategic  objectives  of  the 

organization  are  misleading  at  best  (Cross  and  Lynch,  1988- 

89:24).  If  quality  improvement  is  a  strategic  goal  of  an 

organization,  then  management  indicators  should  be  geared 

toward  measuring  quality  based  on  the  organization's 

definition  of  quality  improvement. 

Quality  indicators  should  be  defined  from  the 

perspective  of  the  customer. 

Define  'on  time'  in  the  customer's  terms  and  customer 
service  as  the  customer  sees  it.  This  applies  not  only 
to  the  external  customers  but  to  the  customers  within 
your  company  who  rely  on  your  product  or  service. 

(Trunick  and  Richardson,  1990:19) 

Knowledge  of  exactly  who  the  customer  is  and  what  the 
customer  needs  is  essential  in  order  to  develop  effective 
management  indicators.  "Clearly,  the  biggest  challenge  for 
quality  managers  now  is  to  determine  what  truly  constitutes 
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customer  satisfaction,  and  to  translate  that  into  measurable 
terms"  (Read  and  Miller,  1991:35). 

Knowledge  of  the  customer  and  their  needs  is  not  the 
only  ingredient  in  meaningful  management  indicators. 
Timeliness  of  information  is  also  crucial  to  the 
effectiveness  of  logistics  measures.  Indicators  that  show 
end  results  of  processes  are  less  effective  than  those  that 
show  the  progress  of  the  individual  process.  End-result 
indicators  tend  to  hide  individual  problem  areas.  For 
example,  the  fact  that  one  process  is  much  worse  than  normal 
could  be  hidden  in  a  report  showing  the  average  performance 
of  many  processes. 

Ad  hoc  information  can  often  be  more  important  to 
managers  than  structured,  scheduled  reports.  "Reports  such 
as  sales  summaries  and  monthly  budget  summaries  may  not  zero 
in  on  current  business  problems”  (Schultheis  and  Sumner, 
1992:656).  These  same  reports  can  be  easily  manipulated  to 
hide  individual  abnormalities  unless  the  indicators  are 
broken  down  to  the  individual  processes,  and  reports  are 
published  as  required,  rather  than  on  a  scheduled  basis. 

Characteristics  of  a  Good  Metric 

AFMC  Metrics  Handbook.  The  Air  Force  Materiel 
Command' s  Metrics  Handbook  provides  eight  attributes  of  a 
good  metric.  They  are: 

1)  It  is  accepted  as  meaningful  to  the  customer 

2)  It  tells  how  well  organizational  goals  and 
objectives  are  being  met  through  processes  and  tasks 
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3)  It  is  simple,  understandable,  logical  and  repeatable 

4)  It  shows  a  trend 

5)  It  is  unambiguously  defined 

6)  Its  data  is  economical  to  collect 

7)  It  is  timely 

8)  It  drives  the  "appropriate  action" 

(AFSC,  1991:2-1) 

ACCEPTED  AS  MEANINGFUL  TO  THE  CUSTOMER  relates  directly  to 
customer  focus  in  that  it  must  be  important  to  the  customer. 
HOW  WELL  GOALS  ARE  BEING  MET  THROUGH  PROCESSES  emphasizes 
not  only  whether  the  goals  are  being  met  but  how  they  are 
being  met  and  how  processes  can  be  improved  to  better  meet 
the  goals.  Measures  should  be  SIMPLE  so  that  they  can  be 
understood  by  everyone  in  the  organization  as  well  as  by  the 
customers  and  REPEATABLE  so  that  processes  can  be  tracked 
over  time  to  better  determine  areas  for  improvement. 

SHOWING  A  TREND  is  a  simple  and  logical  attribute  that 
can  be  easily  applied  and  is  one  of  the  primary  tools  of  TQM 
and  process  improvement.  As  a  matter  of  a  fact,  the  Base 
Supply  Analysis  Section's  primary  job  is  to  keep  track  of 
supply  management  indicators  and  inform  the  Chief  of  Supply 
of  positive  or  negative  trends  during  the  monthly  Chief  of 
Supply  "How  Goes  It."  However,  if  supply  managers  are 
reviewing  non-quality  based  indicators,  then  they  are 
wasting  their  time. 

UNAMBIGUOUSLY  DEFINED  is  very  similar  to  SIMPLE, 
UNDERSTANDABLE,  AND  LOGICAL.  ECONOMICAL  DATA  COLLECTION 
implies  that  the  data  should  be  readily  available.  This 
research  applied  this  concept  because  there  were  already 
enough  processes  being  measured  and  adding  any  more  to  the 
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ones  already  in  the  system  would  be  redundant  and  cost 
prohibitive.  In  order  to  improve  a  process,  TIMELY  measures 
must  be  taken  so  that  problems  can  be  identified  as  they 
occur  and  corrective  action  can  be  taken  immediately. 
Finally,  DRIVING  THE  APPROPRIATE  ACTION  should  result  in 
continuous  process  improvement  and  hopefully  improve 
customer  focus. 

Weaver's  Attributes  for  Quality  Indicators.  Weaver 
introduced  six  characteristics  that  he  considered  important 
in  developing  quality  indicators.  They  are: 

1)  Important 

2)  Easily  understood 

3)  Controlled  by  the  function's  actions 

4)  Evaluate  change 

5)  Use  existing  data 

6)  Measure  both  efficiency  and  effectiveness 

(Weaver,  1991:105) 

Five  of  the  six  characteristics  relate  to  the  customer 
focus  and  continuous  process  improvement  framework  utilized 
in  this  research.  The  first  characteristic,  IMPORTANT, 
relates  to  customer  focus  in  that  it  must  be  important  to 
the  customer.  The  second  characteristic,  EASILY  UNDERSTOOD, 
is  obvious  but  does  not  strongly  correlate  to  either 
customer  focus  or  continuous  process  improvement.  It  may  be 
necessary  to  state  it  as  a  requirement,  but  due  to  its 
intuitiveness,  it  could  be  argued  that  it  does  not  belong  as 
one  of  the  six  most  important  characteristics  for  quality 
measurement.  The  third  characteristic,  CONTROLLED  BY  THE 
FUNCTION'S  ACTIONS,  relates  to  continuous  process 
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improvement.  If  the  organization  does  not  have  control  over 
the  process  being  measured,  then  continuous  process 
improvement  cannot  occur.  The  fourth  characteristic  of 
EVALUATING  CHANGE  also  relates  to  continuous  process 
improvement.  In  order  to  continuously  improve,  an 
organization  must  be  able  to  gauge  progress,  or  in  some 
cases,  retrogression.  As  far  as  USING  EXISTING  DATA,  this 
research  applied  this  concept.  This  characteristic  makes 
sense  because  it  saves  time  and  there  are  already  enough 
processes  being  measured  without  adding  more  to  those 
currently  in  the  system. 

The  final  characteristic  of  MEASURING  BOTH  EFFICIENCY 
AND  EFFECTIVENESS  relates  to  both  customer  focus  and 
continuous  process  improvement.  A  brief  definition  of 
efficiency  can  be  stated  as  a  "relationship  between  inputs 
and  outputs,  or  how  well  the  organization  uses  its  inputs  to 
produce  its  outputs."  (Weaver,  1991:106)  The  organization 
itself  can  keep  track  of  input  to  output  ratio  to  measure 
its  efficiency.  Effectiveness  relates  to  both  customer 
focus  and  continuous  process  improvement.  It  can  be  defined 
as  "the  extent  to  which  an  organization  accomplishes  its 
mission."  (Weaver,  1991:106)  The  customer  will  decide 
whether  the  organization  is  effective. 

Complaints  About  Indicators.  Research  by  Cross  and 
Lynch  showed  that  managers  had  four  major  complaints  about 
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the  way  their  operations  were  being  evaluated.  These 
complaints  can  be  summarized  as  follows: 

1.  Performance  measures  were  not  in  tune  with  the  corporate 
strategy  and  were,  therefore,  yielding  irrelevant  or 
misleading  information  and  provoked  behavior  that 
undermined  achievement  of  their  strategic  objectives. 

2.  These  performance  measures  were  distorting  management's 
understanding  of  how  the  organization  was  operating. 

3.  Traditional  performance  measures  did  not  take  into 
consideration  the  requirements  of  internal  and  external 
customers . 

4 .  Bottom-line  measures  came  too  late  to  make  mid-course 
corrections.  (Cross  and  Lynch,  1988-89:23-24) 

The  first  two  complaints  could  be  alleviated  by 
indicators  that  possessed  the  Metrics  Handbook  attributes  of 
"telling  how  well  goals  are  being  met  through  processes," 
"unambiguously  defined, "  and  "driving  the  appropriate 
action"  and  Weaver's  characteristics  of  "important"  and 
"easily  understood."  An  example  of  the  first  complaint  in 
base  supply  is  the  Releveling  Frequency  measure.  By 
releveling  the  system  too  frequently,  stock  is  ordered  too 
frequently  resulting  in  excess  stock  and  wasting 
transportation  expenses.  A  base  supply  example  of  the 
second  complaint  would  be  looking  at  Stockage  Effectiveness 
ratings  by  themselves  without  taking  Issue  Effectiveness 
into  account.  A  high  stockage  effectiveness  may  give  a 
false  impression  to  management  of  how  well  supply  is 
supporting  the  customer.  A  disparity  between  stockage  and 
issue  effectiveness  can  mean  that  base  supply  is  stocking 
items  that  the  customer  does  not  need. 
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The  third  complaint  deals  mainly  with  the  customer 
focus  aspect  of  the  measure.  Measures  that  are  customer 
focused  and  are  "accepted  as  meaningful  to  the  customer"  can 
alleviate  this  complaint.  By  attempting  to  achieve  an 
extremely  high  inventory  accuracy,  supply  often  inventories 
the  same  stock  more  than  once,  degrading  customer  support 
and  causing  excess  work  for  the  warehouse  personnel.  In 
this  case,  improving  the  inventory  accuracy  by  even  one  or 
two  percentage  points  may  not  be  worth  the  consequences. 

The  final  complaint,  not  having  "timely"  measures,  is 
common  in  base  supply.  This  complaint  can  be  alleviated 
with  indicators  that  "evaluate  change"  instead  of  simply 
reporting  the  after  effects  of  change.  Numerous  monthly 
management  reports,  such  as  the  M-32,  are  heavily  relied 
upon  by  supply  officers  for  management  information.  The 
information  contained  in  the  monthly  reports  often 
highlights  problems  almost  a  month  old. 

Proposed  Measures 

Emmelhainz  provides  two  generic  measures  that  can  be 
applied  in  quality  measurement.  They  are  cycle  time  and 
defects  per  unit  (Emmelhainz,  1991:35).  The  author  strongly 
recommends  that  both  measures  be  used  concurrently.  He  also 
states  that  "A  defect  free  product  or  service  delivered  in  a 
timely  manner  is  precisely  what  customers  want.  If  your 
logistics  organization  can  do  these  two  things,  the  cost  of 
doing  so  will  also  be  the  lowest  possible."  (Emmelhainz, 
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1991:37)  Cycle  time  relates  to  time  factors  which  will  be 
used  to  evaluate  customer  focus  and  continuous  process 
improvement.  Defects  per  unit  relates  to  continuous  process 
improvement.  A  supply  example  could  be  the  Demand 
Processing  section  being  measured  on  the  number  of 
transactions  processed  versus  the  number  of  errors. 

Stickler  presents  six  measures  that  can  be  used  by 
manufacturing  companies  to  move  toward  a  world  class 
organization.  They  are: 

1)  Cycle  time  by  product 

2)  Inventory  turn  by  product 

3)  Set  up  times  on  equipment 

4)  Output /Productivity  by  product  per  person 

5)  Quality-rejected  material 

6)  Suggestion  for  improvements  by  product  per  day  per 
person  (Stickler,  1989:40) . 

At  first  glance,  the  above  measures  appear  to  be  applicable 
only  to  a  manufacturing  company.  However,  five  of  the  six 
measures  can  be  related  to  supply  functions  and  can 
therefore  be  tracked  by  a  base  supply  organization.  SET  UP 
TIMES  ON  EQUIPMENT  is  more  of  a  maintenance  function.  While 
not  a  specific  supply  function,  this  measure  can  be  used  to 
improve  the  organization  by  applying  Emmelhainz'  cycle  time 
and  defects  per  unit  which  will  be  discussed  later.  CYCLE 
TIME  BY  PRODUCT  is  another  factor  that  can  induce  continuous 
process  improvement  if  applied  correctly  using  a  combination 
of  cycle  time  and  defects  per  unit.  INVENTORY  TURN  BY 
PRODUCT  aids  in  keeping  track  of  time,  a  customer  focus 
function,  and  also  measuring  improvement  in  terms  of  time. 
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OUTPUT/PRODUCTIVITY  BY  PRODUCT  PER  PERSON  relates  to 
tracking  continuous  process  improvement. 

If  the  context  of  the  phrase  QUALITY-REJECTED  MATERIAL 
can  include  various  types  of  input  errors,  this  measure  is 
already  being  applied  in  the  base  supply  setting.  Base 
supply  organizations  track  errors  such  as  reverse-posts, 
rejected  keyboard  inputs,  and  warehouse  refusals.  These 
three  errors  are  human  errors  that  should  be  tracked  and  can 
be  continuously  reduced  through  training.  The  sixth 
measure,  SUGGESTION  FOR  IMPROVEMENTS  BY  PRODUCT  PER  DAY  PER 
PERSON,  seems  out  of  place  with  the  first  five 
recommendations.  In  a  broad  sense,  this  indicator  could 
mean  having  a  strong  suggestions  program,  but  the  author  was 
not  specific  in  describing  this  measure.  This  indicator 
probably  has  the  weakest  application  in  a  service-oriented 
organizational  setting.  But,  if  a  suggestion  is  for 
improving  the  supply  "process,"  then  this  measure  could 
definitely  apply. 

Air  Force  Supply  Indicators 

Air  Force  quality  improvement  (TQM)  programs  are 
relatively  new  (DOD,  1988) .  However,  management  indicators 
currently  used  by  base  supply  managers  have  remained 
virtually  unchanged  for  many  years.  The  Supply  Management 
Report  (M-32)  is  the  primary  source  of  supply  indicators 
used  by  base-level  Air  Force  logistics  officers  and  managers 
(DAF,  July  1992:Ch  5,  395).  The  M-32  is  published  monthly 
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in  a  set,  standard  format  and  shows  the  overall  monthly 
status  of  internal  supply  programs  with  little  emphasis  on 
processes.  There  are  at  least  77  indicators  published 
monthly  in  the  M-32,  depending  on  the  number  of  different 
types  of  aircraft  the  base  supports  (Hronek,  March  1993) . 

If  the  M-32  is  being  used  as  supply  management's 
primary  source  of  information  on  the  status  of  base  supply, 
then  Air  Force  logisticians  are  trying  to  implement  quality 
improvement  without  proper  quality  improvement  tools.  These 
past  performance  measures  should  be  realigned  with  the 
objectives  of  quality  improvement. 

Research  by  Greer  and  Moon  identified  six  categories  of 
supply  management  indicators  common  to  Air  Force  supply 
operations : 

1 .  Stockage  Support  Indicators 

2.  Not  Mission  Capable  Supply  (NMCS)  Indicators 

3 .  Priority  Support  Indicators 

4 .  Warehouse  Storage  Indicators 

5.  Repair  Cycle  Indicators 

6.  Computer  Utilization  Indicators  (Greer  &  Moon, 
1981:41) . 

Computer  Utilization  Indicators  are  no  longer  relevant  to 
base  supply  since  base  supply  no  longer  maintains  the 
mainframe  computer. 

A  review  of  AFR  900—14,  USAF  Supply  Effectiveness 
Awards  and  appropriate  guidance  letters,  revealed  that  the 
following  14  indicators  were  being  used  when  evaluating 
bases  for  the  Daedalian  and  Supply  Effectiveness  Awards: 

1.  Mission  Capable  Rate 

2.  Total  Not  Mission  Capable  Supply  Rate 

3.  Bench  Stock  Availability 
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4.  Recoverable  Stockage  Effectiveness  * 

5.  Consumable  Stockage  Effectiveness  * 

6.  Recoverable  Issue  Effectiveness  * 

7.  Consumable  Issue  Effectiveness  * 

8.  Percent  Repairable  This  Station  * 

9.  System  Support  Division  (SSD)  Stockage  Effectiveness 

10.  General  Support  Division  (GSD)  Stockage 
Effectiveness 

11.  SSD  Issue  Effectiveness 

12.  GSD  Issue  Effectiveness 

13.  Item  Records  Past  Due  Inventory 

14.  Inventory  Accuracy  * 

Asterisks  identify  indicators  found  in  the  M-32. 

Telephone  interviews  with  four  CONUS  MAJCOM  supply 
analysts  responsible  for  tracking  indicators  from  their 
subordinate  bases  revealed  that  the  indicators  in  Table  1 
were  being  used.  Three  of  the  four  MAJCOMs  identified  the 
indicators  they  use  when  evaluating  their  subordinate  supply 
accounts.  The  fourth  MAJCOM,  Air  Force  Materiel  Command,  is 
not  currently  reviewing  indicators  due  to  a  recent  major 
reorganization . 

The  indicators  in  Table  1  are  used  most  frequently  by 
four  CONUS  MAJCOMs  to  review  the  status  of  their  subordinate 
supply  accounts.  Stockage  Effectiveness  and  Bench  Stock 
Availability  are  the  only  indicators  common  to  three 
MAJCOMs . 


32 


TABLE  1 

INDICATORS  IDENTIFIED  BY  MAJCOM 


INDICATOR 

ACC 

AETC 

AMC 

1. 

Stockage  Effectiveness  (overall) 

* 

X 

X 

X 

2. 

Issue  Effectiveness  (o/a) 

* 

X 

X 

3. 

Stockage  Effectiveness  (PWS) 

it 

X 

X 

4. 

Issue  Effectiveness  (PWS) 

it 

X 

5. 

Bench  Stock  Availability 

X 

X 

X 

6. 

Item  Records  with  R/O  &  Zero  Acc 

X 

7. 

Retail  Sales  Issue  Eff. 

X 

8. 

Receipt  Not  Due  In  Rate 

it 

X 

X 

9. 

Rex  Code  1,3,4  Rate 

* 

X 

10. 

Releveling  Frequency 

it 

X 

X 

11. 

Follow— yip  Frequency 

it 

X 

X 

12. 

GSD  Orders  vs.  Sales 

X 

13. 

Serv  Balance  W/  No  Whse  Location 

it 

X 

X 

14. 

Warehouse  Refusal  Rate 

X 

X 

15. 

Reverse  Posts 

★ 

X 

16. 

Equipment  Data  Bank 

X 

17. 

Inventory  Accuracy 

X 

*  — 

Indicators  found  in  the  M-32 

During  the  process  of  identifying 

the 

most 

commonly 

used  supply  indicators,  it  was  discovered  that  there  was 
very  little  consensus  among  the  sources  as  to  which 
indicators  were  important.  Only  one  indicator  out  of  28 
(bench  stock  availability)  was  identified  by  all  sources. 
Out  of  the  14  indicators  identified  by  AFR  900-14,  only  two 
were  being  reviewed  by  at  least  one  MAJCOM  (inventory 
accuracy  and  bench  stock  availability) .  Neither  inventory 
accuracy  nor  bench  stock  availability  are  indicators  found 
in  the  M-32 .  Only  nine  of  the  28  indicators  were  reviewed 
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by  more  than  one  MAJCOM.  Overall,  only  ten  of  28  indicators 
(36%)  were  identified  by  more  than  one  source. 

The  research  into  Air  Force  supply  management 
indicators  revealed  that  out  of  at  least  77  indicators  found 
in  the  M-32,  only  eight  indicators  are  used  by  more  than 
half  of  the  four  CONUS  MAJCOMs  when  reviewing  the  status  of 
base  supply  accounts.  One  major  command  found  it  necessary 
to  manually  combine  indicators  to  come  up  with  a  new  metric 
to  meet  their  needs  (Pecoraro,  1993) .  Another  command  is 
using  only  one  M-32  indicator  (Page,  1993) .  These  results 
imply  that  the  current  indicators  may  not  be  meeting  the 
needs  of  the  Air  Force. 

All  indicators  which  were  identified  in  the  Supply 
Effectiveness  Award  evaluation  and/or  by  more  than  one 
MAJCOM  were  retained  for  further  research.  Based  upon  these 
criteria,  the  original  28  indicators  were  narrowed  to  the  18 
indicators  in  Table  2.  The  formulas  for  these  indicators 
may  be  found  in  Appendix  A. 
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TABLE  2 

INDICATORS  SELECTED  FOR  RESEARCH 

indicator' 

1.  Stockage  Effectiveness  (overall) 

2.  Issue  Effectiveness  (overall) 

3.  Stockage  Effectiveness  (primary  weapon  system) 

4.  Mission  Capable  Rate 

5.  Total  Not  Mission  Capable  Supply 

6.  Bench  Stock  Availability 

7.  Recoverable/Consumable  Stockage  Effectiveness 

8.  Recoverable/Consumable  Issue  Effectiveness 

9.  Receipt -Not-Due-In  Rate 

10.  Releveling  Frequency 

11.  Percent  Repairable  This  Station  (overall) 

12.  Follow-up  Frequency 

13.  System  Support  Division  (SSD)  and  General  Support 
Division  (GSD)  Stockage  Effectiveness 

14.  SSD  and  GSD  Issue  Effectiveness 

15.  Serviceable  Balance  With  No  Warehouse  Location 

16.  Warehouse  Refusal  Rate 

17.  Item  Records  Past  Due  Inventory 

18.  Inventory  Accuracy 

*  -  Due  to  the  similarity  of  these  indicators,  they  have 
_ been  treated  as  the  same  for  research  purposes. _ 


Summary 

A  review  of  pertinent  publications  and  interviews  with 
supply  analysts  at  the  four  major  CONUS  MAJCOMs  resulted  in 
identification  of  the  18  most  commonly  used  supply 
indicators  (see  Table  2) .  These  indicators  were  identified 
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in  the  Daedalian  Award  evaluation  and/or  by  more  than  one 
MAJCOM. 

The  relationship  between  the  18  indicators  and  the 
Quality-Based  Metrics  Framework  is  presented  in  Table  3. 
These  ratings  were  given  by  the  authors  based  on  knowledge 
of  the  supply  process,  review  of  the  formulas  for  the 
indicators,  and  analysis  of  the  focus  group  comments.  An 
"S"  implies  a  strong  presence,  a  "W"  implies  a  weak 
presence,  a  "W-"  indicates  a  very  weak  presence,  and  a 
indicates  no  presence  of  the  characteristic  in  the 
indicator. 

The  indicators  support  Continuous  Improvement  to  the 
greatest  extent.  This  implies  that  supply  indicators  are 
primarily  geared  toward  internal  measurement.  The 
indicators  strong  in  continuous  improvement  reflect 
effectiveness  of  training.  Customer  Focus  is  well  supported 
by  only  five  of  the  indicators.  Two  of  these  indicators, 
MICAP  Rate  and  TNMCS  are  mission  focused.  As  evident  in  the 
table,  none  of  the  indicators  measure  time  and  only  one. 
Inventory  Accuracy,  strongly  measures  cost,  even  though 
supply  organizations  manage  millions  of  dollars  in  assets. 

The  importance  of  Customer  Focus  and  Continuous  Process 
Improvement  on  TQM  cannot  be  overemphasized.  The  need  to 
incorporate  the  supporting  elements  of  time  and  cost  into 
metrics  is  readily  apparent.  Emmelhainz,  Stickler,  Weaver, 
and  The  Metrics  Handbook  all  emphasize  the  importance  of 
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TABLE  3 


RELATIONSHIP  BETWEEN  ORIGINAL  INDICATORS  AND  FRAMEWORK 


INDICATOR 

CF 

Cl 

Time 

Cost 

Stockage  Effectiveness  (overall) 

- 

w 

- 

w 

Issue  Effectiveness  (overall) 

W 

w 

- 

w 

Stk  Eff  (primary  weapon  system) 

S 

w 

- 

w 

Mission  Capable  Rate  (MICAP) 

s 

w 

- 

- 

Total  Not  Mission  Capable  Supply 

s 

s 

- 

- 

Bench  Stock  Availability 

s 

w 

— 

- 

Recoverable/Consumable  Stk  Eff 

- 

w- 

- 

w- 

Recoverable/Consumable  Iss  Eff 

W- 

w- 

- 

w- 

Receipt-Not-Due-In  Rate 

w- 

w 

- 

— 

Releveling  Frequency 

- 

- 

- 

- 

Percent  Repairable  This  Station 

w 

w 

- 

w- 

Follow-up  Frequency 

- 

- 

- 

- 

SSD/GSD  Stockage  Effectiveness 

- 

w- 

- 

w- 

SSD/GSD  Issue  Effectiveness 

w- 

w- 

- 

w- 

Serv  Balance  w/No  Warehouse  Loc 

w 

s 

- 

w 

Warehouse  Refusal  Rate 

s 

s 

- 

w 

Item  Records  Past  Due  Inventory 

- 

s 

- 

M 

Inventory  Accuracy 

w 

s 

- 

s 
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time  factors  and  the  timeliness  of  measures.  The  importance 
of  cost  factors  is  ever  increasing  due  to  the  shrinking 
defense  budget,  the  need  for  the  DOD  to  operate  more 
"business-like,"  and  the  emphasis  on  inventory  reduction. 

The  Quality-Based  Metrics  Framework  is  not  readily 
apparent  in  supply  management  indicators .  The  lack  of 
appropriate  measures  for  supply  organizations  hampers  the 
effectiveness  and  efficiency  of  the  entire  Air  Force. 

Chapter  3  will  present  the  method  used  to  develop 
supply  indicators  based  on  the  Quality-Based  Metrics 
Framework.  The  steps  taken  to  verify  and  validate  the 
indicators  will  also  be  presented. 
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Ill .  Methodology 


The  methodology  presented  in  this  chapter  was  used  to 
answer  the  investigative  questions  posed  in  Chapter  1 . 

These  four  questions  were  answered  through  the  procedures 
outlined  in  this  chapter.  The  first  two  questions  were 
answered  through  a  combination  of  literature  searches  and 
personal  interviews.  The  second  two  questions  were  answered 
by  presenting  information  to  senior  supply  managers  through 
a  Group  Support  System. 

Desired  Characteristics 

Investigative  question  #1: 

According  to  the  TQM  philosophy,  what  are  the 
appropriate  characteristics  which  management  indicators 
should  possess  in  order  to  assess  the  performance  of 
base  supply? 

A  literature  review  was  conducted  to  identify  the 
characteristics  that  a  indicator  should  possess  under  the 
TQM  philosophy.  This  literature  review  covered  TQM 
philosophies  both  within  the  federal  government  and  the 
private  sector.  This  investigation  determined  that  the 
major  characteristics  performance  indicators  should  possess 
in  accordance  with  the  TQM  philosophy  are  1)  a  customer/ 
mission  focus  and  2)  an  indication  of  how  to  improve  the 
supply  process. 
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Common  Indicators 


Investigative  question  #2 : 

What  are  the  most  commonly  used  supply  management 

indicators? 

In  order  to  determine  the  most  commonly  used  supply 
management  indicators,  an  initial  review  of  past  AFIT  Theses 
was  conducted  to  determine  indicators  that  were  considered 
important.  Next,  a  review  of  Air  Force  publications  on  the 
USAF  Supply  Effectiveness  Awards  identified  indicators  that 
are  evaluated  by  the  Air  Staff  when  selecting  the 
outstanding  supply  unit  of  the  year.  Finally,  senior  supply 
analysts  at  the  four  CONUS  MAJCOMs  were  interviewed  via 
telephone.  Three  of  the  four  identified  which  indicators 
they  used  when  evaluating  their  subordinate  supply  accounts. 
The  fourth  MAJCOM,  Air  Force  Materiel  Command,  is  not 
currently  reviewing  any  indicators  from  subordinate  supply 
accounts  due  to  a  recent  major  reorganization. 

All  indicators  which  were  identified  in  the  Supply 
Effectiveness  Award  evaluation  criteria  or  by  more  than  one 
MAJCOM  were  used.  The  list  of  indicators  and  where  they 
were  identified  are  listed  in  Table  4.  The  total  number  of 
indicators  used  in  this  research  was  18  (see  Table  5  and 
Appendix  A) . 
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TABLE  4 


INDICATORS  IDENTIFIED  BY  AT  LEAST  ONE  SOURCE 


INDICATOR 


SOURCE 

AFR  900-14  ACC  AETC  AMC 


Stockage  Effectiveness 
(overall) 

Issue  Effectiveness  (o/a) 
Stockage  Effectiveness 
(primary  weapon  system) 
Issue  Effectiveness 
(primary  weapon  system) 


Mission  Capable  Rate  X 

Total  Not  Mission  Capable  X 

Supply  (TNMCS)  Rate 
Bench  Stock  Availability  X 

Item  Records  with  R/O  & 

Zero  Acc 

Recoverable  Stockage  Eff.  X 

Consumable  Stockage  Eff.  X 

Retail  Sales  Issue  Eff. 
Recoverable  Issue  Eff.  X 

Consumable  Issue  Eff.  X 

Receipt  Not  Due  In  Rate 
Rex  Code  1,3,4  Rate 
Releveling  Frequency 
%  Repairable  This  Station  X 

Follow-up  Frequency 
System  Support  Division  (SSD)  X 
Stockage  Effectiveness 
SSD  Issue  Effectiveness  X 

General  Support  Division  X 

(GSD)  Stockage  Effectiveness 
GSD  Issue  Effectiveness  X 


GSD  Orders  vs.  Sales 
Serviceable  Balance  With 
No  Warehouse  Location 
Warehouse  Refusal  Rate 
Item  Records  Past  Due  X 

Inventory 
Reverse  Posts 
Equipment  Data  Bank 
Inventory  Accuracy  X 


X  X 

X 

X 

X 


XXX 

X 


X 


X  X 

X 

X  X 

X  X 


X 

X  X 

X  X 


X 

X 

X 


X 

X 

X 
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TABLE  5 


INDICATORS  PRESENTED  TO  FOCUS  GROUP 


INDICATOR 

1.  Stockage  Effectiveness  (overall) 

2.  Issue  Effectiveness  (overall) 

3.  Stockage  Effectiveness  (primary  weapon  system) 

4 .  Mission  Capable  Rate 

5.  Total  Not  Mission  Capable  Supply 

6.  Bench  Stock  Availability 

7.  Recoverable/Consumable  Stockage  Effectiveness 

8.  Recoverable/Consumable  Issue  Effectiveness 

9 .  Receipt -Not-Due-In  Rate 

10.  Releveling  Frequency 

11.  Percent  Repairable  This  Station  (overall) 

12.  Follow-up  Frequency 

13.  System/General  Support  Division  (SSD  and  GSD)  Stockage 
Effectiveness 

14.  System/General  Support  Division  (SSD  and  GSD)  Issue 
Effectiveness 

15.  Serviceable  Balance  With  No  Warehouse  Location 

16.  Warehouse  Refusal  Rate 

17.  Item  Records  Past  Due  Inventory 

18.  Inventory  Accuracy 

*  -  Due  to  the  similarity  of  these  indicators,  they  have 
been  treated  as  the  same  for  the  purpose  of  this 
research. 


Indicator  Usefulness 

Investigative  question  #3: 

In  terms  of  TQM,  how  useful  are  these  indicators  to 
base  supply  officers? 

A  focus  group  is  defined  as  a  panel  of  respondents  led 
by  a  moderator.  The  moderator  uses  principles  of  group 
dynamics  to  focus  or  guide  the  group  in  an  exchange  of 
ideas,  feelings,  and  experiences  on  a  clearly  understood 
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topic  (Emory  and  Cooper,  1991:147).  A  focus  group  was  used 
to  evaluate  the  usefulness  of  these  indicators,  in  terms  of 
TQM,  to  supply  managers  using  a  modified  Nominal  Group 
Technique.  The  focus  group  was  decided  upon  because  a  great 
deal  of  research  has  shown  that  decisions  made  by  five  or 
more  participants  are  superior  to  individual  decision  making 
(Gibson  and  otters,  1991:585;  Meister,  1985:318;  Hare, 
1982:329) . 

Nominal  Group  Technique.  The  technique  used  in  this 
portion  of  the  research  is  a  modified  Nominal  Group 
Technique  (NGT)  method  of  effecting  group  decision-making. 
NGT  is  defined  as  a  "Technique  that  promotes  creativity  by 
bringing  people  together  in  a  very  structured  meeting  that 
allows  little  verbal  communication.  Group  decision  is  the 
mathematically  pooled  outcome  of  individual  votes"  (Gibson 
and  others,  1991:589).  A  NGT  process  consists  of  four 
steps : 

1.  Silent  judgments  by  individuals  in  the  presence  of 
the  group . 

2.  Presentation  to  the  group  of  all  individual 
judgments  without  discussion. 

3.  Group  discussion  of  each  judgment  for  clarification 
and  evaluation. 

4 .  Individual  reconsideration  of  judgments  and 
mathematic  combination.  (Meister,  1985:319) 

This  research  modifies  the  NGT  as  follows: 

1.  Anonymous  open  discussion  by  all  participants  via 
the  GSS  network. 
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2.  Open  verbal  group  discussion  of  alternatives. 

3.  Individual  consideration  and  judgment  of  all 
alternatives  with  previous  comments  available  for 
review. 

4.  Mathematic  combination  of  judgments. 

These  modifications  were  necessary  to  more  effectively 
present  information  and  gather  data  using  the  GSS  network 
(Wolfe,  1993) . 

Group  Support  System.  A  Group  Support  System  (GSS)  was 

used  to  facilitate  the  modified  NGT  process.  A  GSS  is 

different  from  a  Group  Decision  Support  System  (GDSS)  in 

that  the  GDSS  is  usually  an  on-line  system  that  is  linked 

with  large  data  bases  so  that  a  group  of  individuals  have 

access  to  the  same  data.  Group  Support  Systems  are: 

...computer-based  systems  that  provide  a  variety  of 
tools  to  facilitate  the  meeting  process.  In  part  these 
systems  are  electronic  implementations  of  older  methods 
-  e.g.  Delphi  and  NGT  -  that  have  been  used  to  improve 
the  quality  of  meetings  over  the  last  30  years. 

(Armstrong  Laboratory) 

The  GSS  used  in  this  research  is  located  in  the 
Armstrong  Laboratory,  Area  B,  Wright-Patterson  AFB,  OH.  The 
Logistics  Research  Division  of  the  Armstrong  Laboratory 
performs  research  on  technology  for  improving  performance 
with  integrated  systems.  This  GSS  is  part  of  another  on¬ 
going  research  project  on  the  potential  of  Group  Support 
Systems  (Armstrong  Laboratory) .  The  software  used  in  this 
research  is  GroupSvstems  V  by  Ventana  Corporation  (copyright 
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1990-1992,  Ventana  Corporation,  copyright  1988-1989  MIS 
Department,  University  of  Arizona) . 

Focus  Group  Selection. 

Population  of  Interest.  The  population  of 
interest  for  the  members  of  the  focus  group  is  the  955 
United  States  Air  Force  commissioned  officers  currently 
serving  in  all  64XX  Air  Force  Specialty  Codes  (AFSCs) (Egge, 
1992) . 

Sample .  There  were  11  volunteers  for  the  focus 
group.  The  actual  focus  group  consisted  of  five  members, 
due  to  scheduling  constraints.  Research  has  shown  that  this 
number  is  appropriate  when  using  the  Nominal  Group  Technique 
(Gibson  and  others,  1991:589;  Hare,  1982:142;  Hare, 

1976:231;  Jewell  and  Reitz,  1981:86).  A  judgment  sampling 
technique  was  used  to  select  officers  from  the  population  of 
interest.  The  criteria  for  inclusion  in  the  focus  group 
included: 

-  members  must  have  at  least  ten  years  of  Air  Force 
supply  experience. 

-  members  must  hold  the  rank  of  Captain,  Major, 
Lieutenant  Colonel,  or  Colonel. 

-  members  must  be  familiar  with  the  M-32  Supply 
Management  Report . 

-  members  must  have  prior  TQM  training. 

-  members  must  voluntarily  participate. 

Indicator  Evaluation.  A  focus  group  was  used  to 
evaluate  the  indicators  identified  in  the  second 


45 


investigative  question  (Table  5)  through  the  modified  NGT. 
The  focus  group  met  at  the  GSS  room  in  the  Armstrong 
Laboratory  where  they  were  given  an  initial  briefing  on  the ' 
GSS  by  the  laboratory  staff.  Periodic  breaks  were  taken  to 
help  prevent  fatigue  and  eye  strain. 

Dry  Run.  After  the  initial  briefing,  the  research 
team  explained  to  the  members  of  the  focus  group  the  purpose 
of  the  research  and  the  procedures  to  be  followed  while 
using  the  modified  NGT.  At  this  time,  three  supply 
indicators  that  were  not  among  any  of  the  18  identified  in 
Table  5  were  used  to  familiarize  the  focus  group  members  on 
the  computer  system  and  the  NGT.  The  following  procedures 
were  used  to  conduct  the  dry  run  and  the  actual  evaluation. 

Actual  Evaluation.  Once  the  dry  run  was 
completed,  the  actual  indicators  under  study  were  presented 
to  the  group  for  evaluation  using  the  Topic  Commenter 
software  option  (see  Appendix  A) . 

First,  all  the  supply  indicators  were  displayed  on  each 
individual  screen.  Each  of  the  18  indicators  was  displayed 
on  its  own  commenter  card.  There  were  19  cards  displayed. 
The  first  18  contained  the  indicators  from  Table  5  and  the 
last  card  was  left  for  generic  comments. 

The  researchers  then  asked  the  group  to  make  comments 
about  the  first  five  indicators  (only)  in  relation  to  the 
following  two  questions:  "How  well  does  this  indicator 
reflect  customer/mission  support?"  and  "How  well  does  this 
indicator  lead  you  to  continually  improve  the  process?"  The 
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comments  were  limited  to  the  first  five  indicators  so  that 


no  one  would  get  too  far  ahead  of  the  group. 

Without  verbal  discussion,  each  individual  then  typed 
comments  about  the  indicator  in  relation  to  two  questions 
asked  by  the  researchers .  The  GSS  recorded  and  collated  the 
comments  and  displayed  them  on  an  overhead  viewing  screen 
anonymously,  as  they  were  entered.  Once  all  participants 
commented  and  electronically  replied  to  the  comments  on  each 
indicator,  the  research  team  led  the  group  in  a  structured 
discussion  on  each  comment  so  that  the  comment  could  be 
clarified,  if  necessary,  and  examined  by  the  entire  group. 

At  this  point,  a  break  was  taken.  After  the  break,  the 
above  process  was  repeated  for  the  next  group  of  five 
indicators.  This  continued  until  all  indicators  were 
covered. 

After  all  of  the  indicators  had  been  covered,  each 
participant  was  asked  to  make  any  comments  relating  to 
supply  management  indicators,  how  to  improve  them,  and 
possible  alternative  indicators  on  the  last  commenter  card. 
These  comments  were  used  to  help  modify  indicators  in  order 
to  answer  the  fourth  investigative  question. 

After  all  indicators  were  commented  upon  and  discussed, 
participants  rated  each  indicator  on  how  well  it  reflected 
the  level  of  customer  support  and  how  well  it  would  lead 
them  to  continually  improve  the  supply  process.  The  ratings 
were  on  a  scale  of  1  to  10,  with  1  meaning  little  or  no 
indication  of  the  level  of  customer  support/continuous 
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improvement  and  10  being  an  extremely  high  indication  of  the 
level  of  customer  support /continuous  improvement.  The  1  to 
10  scale  was  used  as  that  is  the  only  rating  scale  supported 
by  the  software. 

The  software  allowed  participants  to  abstain  from 
voting  on  any  indicator.  To  ensure  more  accurate  data 
collection,  participants  were  allowed  to  abstain  from  voting 
on  any  indicator  with  which  they  were  unfamiliar. 

After  the  data  collection  process  was  complete,  the 
mean,  standard  deviation,  and  median  of  the  ratings  on  each 
indicator  was  computed  for  each  criterion  (customer/mission 
focus  and  process  improvement) .  These  numbers,  along  with 
the  individual  participant  ratings,  were  listed  in  the  order 
in  which  they  were  presented  (see  Figure  2  and  Appendix  C) . 
Finally,  all  indicat  c.rs  were  rank  ordered  by  the  median 
rating  for  each  criterion  and  by  the  sum  of  the  two  criteria 
ratings .  The  mean  and  standard  deviation  were  used  to  break 
any  ties  (see  Appendix  C) . 

The  median  was  decided  upon  rather  than  the  mean 
because  the  median  is  less  sensitive  than  the  mean  in  small 
measurements  and  is  a  better  measure  of  central  tendency  in 
this  situation  (McClave  and  Benson,  1991:89). 

The  above  information  on  each  indicator  was  analyzed 
for  possible  modification  and  improvement  of  the  indicators 
in  accordance  with  the  fourth  investigative  question. 
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1.  STOCKAGE  EFFECTIVENESS  -  (OVERALL) 


Participant  Ratings 

*  Criteria  *123456789  10  NnMNSDMD 


Customer/  -  -  -  -  2  -  2  1  -  -  5  56.41.37.0 

Mission  Focus 

Continuous  1  1  -  -  -  1  2  -  -  -  5  54.62.86.0 

Process  Improvement 

2.  ISSUE  EFFECTIVENESS  -  (OVERALL) 


Participant  Ratings 

*  Criteria  *123456789  10  NnMNSDMD 


Customer/  —  —  —  —  1  1  1  —  2  —  5  57.21.77.0 

Mission  Focus 

Continuous  —  1  1  —  1  —  1  —  1  —  5  55.22.95.0 

Process  Improvement 

Figure  2 .  Partial  List  of  Indicator  Ratings 


Modifying  Indicators 

Investigative  question  #4: 

Can  these  supply  indicators  be  modified  so  that  they 

are  more  in-line  with  the  TQM  philosophy? 

The  researchers  attempted  to  modify  all  indicators  to 
bring  them  more  in  line  with  the  TQM  philosophy.  Any 
indicator  with  median  ratings  of  eight  or  greater  on  each 
criteria  is  strong  in  both  customer  focus  and  process 
improvement.  These  indicators  were  considered  appropriate 
indicators  to  use  according  to  the  TQM  philosophy  and  were 
retained  along  with  their  modified  versions,  if  they  were 
modified. 

The  process  of  modifying  the  indicators  consisted  of: 

1 .  Identifying  the  source  data  from  which  the 
indicators  are  derived. 
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2.  Applying  content  analysis  to  analyze  the  inputs  from 
the  focus  group. 

3.  Analyzing  the  source  data  and  focus  group  inputs  to 
see  whether  it  was  feasible  to  modify  the  current 
indicators . 

The  indicator  sources  were  found  in  investigative 
question  #2.  This  was  primarily  the  M-32  report.  Those 
indicators  not  found  in  the  M-32  report  were  identified  by 
the  researchers  through  AFM  67-1  and  interviews  with  supply 
analysts . 

Focus  group  comments  were  examined  using  content 
analysis.  The  researchers  separately  analyzed  each  comment 
made  by  the  group  and  rated  the  comment  as  positive, 
negative,  or  neutral  as  the  comment  related  to  the  aspects 
of  customer  focus  or  process  improvement.  The  following 
formula  was  used  to  rate  the  intercoder  reliability  (IR)  of 
the  researchers : 

IR  _  Number  of  Agreements 

Total  Number  of  Comments 

(Miles  and  Huberman,  1984:63) 

After  the  intercoder  reliability  was  calculated,  the 
researchers  discussed  the  comments  and  reevaluated  their 
ratings.  The  intercoder  reliability  was  recalculated  as 
recommended  by  the  text,  and  the  reliability  was  found  to  be 
extremely  high. 

In  order  to  modify  the  indicators,  the  focus  group 
comments,  the  characteristics  identified  in  the  first 
investigative  question,  and  Juran's  quality  formula  were 
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applied  wherever  possible.  Juran' s  quality  formula  is  as 
follows : 


QUALITY 


Frequency  of  Deficiencies 
Opportunities  for  Deficiencies 


(Juran,  1989:200). 

The  above  formula  is  simply  applying  ratios  to  measures  in 
order  to  establish  a  baseline  for  continuous  process 
improvement.  An  example  of  a  supply  application  of  this 
formula  is  the  Receipt-Not-Due-In  Rate.  The  numerator  would 
be  "number  of  receipts  without  a  corresponding  due-in, " 
while  the  denominator  would  be  "total  number  of  receipts . " 

By  using  only  the  number  of  receipts-not-due-in  without 
considering  the  opportunities  for  deficiencies,  there  is  no 
baseline  for  trend  analysis  or  comparison  between 
organizations . 

As  a  result  of  the  above  process,  three  indicators  were 
not  modified,  eight  were  modified  versions  of  original 
indicators,  two  were  current  supply  indicators  that  were  not 
any  of  the  original  indicators,  and  five  were  new  indicators 
developed  by  the  researchers . 


Quality  Improvement .  In  order  to  determine  whether  the 
revised  indicators  were  an  improvement  over  the  original 
indicators,  a  second  focus  group  meeting  was  conducted. 

This  focus  group  consisted  of  the  same  sample  as  the  first 
focus  group  in  order  to  reduce  potential  difference  bias. 
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Indicator  Evaluation.  The  focus  group  was  used  to 


evaluate  the  indicators  modified  by  the  research  team.  The 
focus  group  used  the  GSS  at  Armstrong  Laboratory  to  evaluate 
the  revised  indicators.  The  individuals  were  allowed  to 
evaluate  the  indicators  at  different  times.  The  same 
questions  were  posed  and  members  used  the  same  scale  to  rate 
the  revised  indicators. 

The  ratings  of  both  focus  group  meetings  were  compared 
to  determine  if  the  revised  indicators  more  fully  met  the 
requirements  for  useful  indicators  in  terms  of  TQM. 

This  was  accomplished  by  conducting  a  one-tailed  t-test 
to  determine  if  the  revised  indicators  were  significantly 
better  than  the  original  indicators.  The  average  median 
rating  of  the  indicators  from  the  first  focus  group  meeting 
was  compared  with  the  average  median  rating  of  the  revised 
indicators  from  the  second  focus  group  meeting.  An  alpha 
(a)  of  .05  was  used. 

The  Test  of  Hypothesis  was  conducted  as  follows: 

H0:  No  difference  between  the  average  medians  of  the  two 

focus  group  meetings. 

Ha:  The  average  median  of  the  second  focus  group  meeting  is 

greater  than  that  from  the  first. 


Test  Statistic :  t  = 

<-F> 

yfn 

Reject  region :  t  >  ta,  where  ta  has  n  -  1  df. 
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Where  ^  is  the  average  median  score  of  the  indicators  from 
the  first  focus  group  meeting  and  |i2  is  the  average  median 
score  from  the  second  focus  group  meeting.  The  results  of 
this  test  verified  whether  or  not  the  revised  indicators 
were  an  improvement  over  the  original  indicators  in  terms  of 
TQM. 

Using  statistical  and  content  analysis,  a  final  group 
of  indicators  was  produced.  This  group  of  indicators 
underwent  a  double  screening  process  by  experienced  supply 
officers  and  should  be  useful  metrics.  In  addition,  these 
metrics  were  evaluated  by  a  larger  sample  of  field  grade 
supply  officers  throughout  the  Air  Force  to  validate  these 
findings . 

Validation.  In  order  to  determine  the  validity  of 
these  indicators  throughout  the  Air  Force,  questionnaires 
were  sent  to  all  field  grade  supply  officers  at  the  CONUS 
MAJCOMs,  the  Air  Force  Logistics  Management  Agency,  and  the 
Standard  Service  Center.  All  officers  met  the  same  criteria 
as  those  in  the  focus  group.  These  questionnaires  contained 
the  final  18  metrics  as  determined  by  the  researchers 
through  the  two  focur  oup  meetings  (see  Appendix  G) . 

The  officers  rated  each  indicator  on  a  scale  of  1  to  10 
on  the  same  two  criteria  as  the  focus  group.  The  ratings 
from  the  questionnaires  were  compared  to  determine  if  the 
results  from  the  focus  group  were  consistent  across  the  Air 
Force . 


53 


Due  to  different  sample  sizes,  the  researchers  used 
non-parametric  statistics  to  compare  the  results  of  the 
survey  ratings  with  the  focus  group  results.  The  indicators 
were  ranked  from  highest  to  lowest  median  with  ties  being 
broken  in  favor  of  the  highest  mean  and  if  necessary,  lowest 
standard  deviation. 

The  Wilcoxon  Rank  Sum  Test  was  used  to  determine  which 
group  of  officers  rated  the  indicators  higher.  Focus  group 
ratings  equal  or  lower  than  the  ratings  from  the 
questionnaires  sent  to  the  sample  of  supply  officers 
throughout  the  Air  Force  implies  potential  Air  Force-wide 
application. 

The  Test  of  Hypothesis: 

H0:  The  probability  distributions  corresponding  to  the 

focus  group  and  the. sample  are  identical. 

Ha:  The  probability  distribution  corresponding  to  the 

sample  lies  to  the  right  or  left  of  the  probability 
distribution  of  the  focus  group. 

Test  Statistic:  The  rank  sum  T,  associated  with  the  sample. 
Reject  Region:  Reject  if  T  <  TL  or  T  >  T0  for  a  =  .025 

Spearman's  Rank  Correlation  Coefficient  was  applied  to 
determine  if  the  focus  group  and  the  sample  rated  the 
indicators  in  a  similar  order. 

The  Test  of  Hypothesis: 

H0:  There  is  no  population  correlation  between  ranks. 

Ha:  There  is  a  population  correlation  between  ranks. 
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6  Sdf 


Test  Statistic :  r  =  1  - 

s  n(n2  -  1) 


Reject  Region:  zB  <  -rg  «  or  rs  >  rs  « 

'2  '2 


Summary 

This  chapter  outlined  the  methodology  to  be  used  in 
order  to  answer  the  investigative  questions  posed  in  chapter 
one.  Specifically,  the  desired  characteristics  for 
management  indicators  were  found  through  an  in-depth 
literature  review.  A  determination  of  commonly  used  supply 
management  indicators  was  found  through  a  literature  review 
and  telephone  interviews  with  MAJCOM  supply  analysts .  A 
determination  of  the  usefulness  of  current  indicators  in 
terms  of  TQM  was  made  by  applying  a  modified  Nominal  Group 
Technique  using  a  GSS.  A  modification  of  indicators  was 
made  by  analyzing  the  focus  group  comments  through  content 
analysis  and  applying  those  comments  and  the  desired 
characteristics  to  the  original  indicators.  The  revised 
indicators  were  then  compared  to  the  original  indicators 
using  the  GSS.  Finally,  the  results  were  validated  Air 
Force-wide  through  questionnaires  to  the  major  commands. 

The  next  chapter  will  analyze  the  results  of  this 
research . 
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IV.  Findings  and  Analysis 


Overview 

The  purpose  of  this  research  is  to  determine  which 
management  indicators  should  be  used  in  order  to  provide 
supply  managers  with  quality  measurement  tools.  This 
chapter  focuses  on  the  findings  and  analysis  of  the  research 
used  to  answer  the  third  and  fourth  investigative  questions: 

-  In  terms  of  TQM,  how  useful  are  these  indicators  to 
base  supply  officers? 

—  Can  these  supply  indicators  be  modified  so  that 
they  are  more  in-line  with  the  TQM  philosophy? 

In  order  to  answer  the  third  investigative  question, 
the  18  indicators  selected  for  research  were  presented  to  a 
focus  group  of  highly  experienced  supply  officers  (see  Table 
2) .  The  group  rated  the  indicators  and  commented  on  their 
usefulness  in  terms  of  customer  focus  and  continuous  process 
improvement . 

Content  analysis  was  used  to  analyze  the  comments  and 
suggestions  made  by  the  focus  group.  This  content  analysis 
consisted  of  the  authors  separately  rating  each  comment  as 
having  a  positive,  negative,  or  neutral  referential  theme. 
Intercoder  reliability  was  then  calculated  at  67.0  percent. 
The  authors  then  discussed  each  comment  and  repeated  the 
rating  process.  The  final  intercoder  reliability  was  89.4 
percent.  The  general  theme  of  the  comments  was  matched 
against  the  overall  ratings  for  each  indicator  in  order  to 
validate  the  ratings.  No  blatant  inconsistencies  were 
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found.  This  analysis,  combined  with  the  framework  from  the 
first  question,  was  used  to  modify  the  18  indicators  per  the 
fourth  investigative  question. 

The  revised  indicators  were  presented  to  the  same  focus 
group  for  comments  and  ratings.  The  results  of  the  two 
meetings  were  compared  using  a  t-test  to  verify  whether  the 
revised  indicators  were  significantly  better  than  the 
original  18. 

A  final  validation  of  the  research  was  performed  by 
sending  questionnaires  to  all  field  grade  supply  officers  at 
the  CONUS  MAJCOMs,  the  Air  Force  Logistics  Management 
Agency,  and  the  Standard  Service  Center.  These  officers 
rated  the  final  indicators  and  the  ratings  were  compared 
throughout  the  Air  Force. 

Usefulness  of  Current  Indicators 

A  focus  group,  consisting  of  five  senior  supply 
officers,  evaluated  the  indicators  found  in  investigative 
question  #2.  These  officers  had  an  average  of  18.4  years  of 
supply  experience  (low  14,  high  24),  and  all  but  one  had 
been  a  Chief  of  Supply  (COS)  at  least  once.  Three  had  been 
COS  twice. 

Analysis  of  Comments.  Based  on  the  results  from 
investigative  question  #1,  the  focus  group  was  asked  to 
comment  and  rate  the  original  18  indicators  in  terms  of  how 
well  each  indicator  led  to  continuous  process  improvement 
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and  how  well  they  indicate  the  level  of  customer  support. 

The  original  comments  and  ratings  are  in  Appendices  B  and  C. 
.  (See  Table  6  for  the  sum  median  and  mean  ratings.) 


TABLE  6 

SUM  MEDIAN  AND  MEAN  OF  ORIGINAL  INDICATORS 
(CUSTOMER  FOCUS  +  CONTINUOUS  PROCESS  IMPROVEMENT) 


INDICATOR  1 

MEDIAN 

MEAN 

Warehouse  Refusal  Rate 

Total  Not  Mission  Capable  Supply 

17.0 

16.4 

(TNMCS) 

16.0 

16.0 

Bench  Stock  Availability 

15.0 

14.0 

Inventory  Accuracy 

Stockage  Effectiveness  (Primary 

15.0 

12.8 

Weapon  System) 

14.0 

14.6 

Stockage  Effectiveness  (overall) 

13.0 

11.0 

Mission  Capable  Rate  (MI CAP) 

12.0 

13.0 

Issue  Effectiveness  (overall) 

12.0 

12 . 4 

Serviceable  Balance  w/No  Whse  Loc 

11 . 0 

9.4 

Item  Records  Past  Due  Inventory 

10.0 

9.6 

Percent  Repairable  This  Station 

10.0 

9.2 

Receipt-Not-Due-In  Rate 

8.0 

9.0 

Recoverable  (Consumable)  Stk  Eff 

7.0 

7.8 

Recoverable  (Consumable)  Iss  Eff 
System  Support  Division/General 

6.0 

7.6 

Support  Division  (SSD/GSD)  Stk  Eff 
System  Support  Division/General 

6.0 

7.0 

Support  Division  (SSD/GSD)  Iss  Eff 

5.0 

6.8 

Releveling  Frequency 

4.0 

5.8 

Follow-Up  Frequency 

4.0 

4.4 
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Many  comments  revealed  how  these  indicators  are  not 
customer  focused.  Comments  related  to  the  four  Stockage 
Effectiveness  indicators  (Overall,  Primary  Weapon  System, 
Recoverable/Consumable,  and  SSD/GSD)  implied  that  these 
indicators  are  not  customer  oriented  because  these  measures 
exclude  items  that  are  not  stocked  but  are  still  needed  by 
customers.  Many  of  the  other  indicators  were  viewed  as  low- 
level,  internal  measures  of  very  little  importance  to  the 
customers.  These  indicators  were  measures  of  how  often 
supply  personnel  were  accomplishing  routine  tasks  and 
measures  of  supplier  support,  limited  by  imposed  fiscal 
constraints.  This  type  of  indicator  included  Follow-Up 
Frequency,  Releveling  Frequency,  SSD/GjD  indicators,  and 
Repairable/Consumable  indicators . 

Some  indicators  were  viewed  as  being  more  customer 
focused.  Issue  Effectiveness  indicators  can  show  how  well 
supply  anticipates  customer  requirements  because  they 
include  items  that  are  not  stocked  in  their  calculation. 
Customers  do  not  care  whether  or  not  supply  is  authorized  to 
stock  an  item.  Bench  Stock  Availability  and  Warehouse 
Refusal  Rate  both  were  commented  upon  as  high  in  customer 
focus.  They  are  both  direct  measures  of  how  well  supply 
supports  the  customer.  MICAP  and  TNMCS  rates  are  measures 
that  show  how  well  supply  supports  the  mission  and, 
therefore,  received  many  positive  comments. 

Comments  revealed  that  many  of  the  indicators  do  not 
lead  to  continuous  process  improvement.  The  four  Stockage 
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Effectiveness  indicators  are  primarily  controlled  by  fiscal 
constraints  and  depot  support.  The  current  formulas  leave 
room  for  potential  manipulation  by  supply  to  artificially 
inflate  these  indicators.  Neither  Releveling  Frequency  nor 
Follow-Up  Frequency  leads  to  continuous  process  improvement 
because  they  both  deal  with  frequency  of  accomplishing 
rather  routine  tasks.  Furthermore,  if  releveling  and 
follow-up  are  done  too  often  by  overriding  the  computer 
programs,  overstocking  of  unneeded  items  and  inefficient  use 
of  funds  can  occur. 

A  number  of  indicators  were  viewed  as  supporting 
continuous  process  improvement.  Receipt-Not-Due-In  Rate, 
Warehouse  Refusal  Rate,  and  Inventory  Accuracy  all  drive 
continuous  process  improvement,  as  they  are  good  indicators 
of  possible  larger  problems.  This  will  lead  supply  managers 
to  conduct  additional  research  to  find  the  underlying  cause 
of  the  problem.  Item  Records  Past  Due  Inventory, 

Serviceable  Balance  With  No  Warehouse  Location,  and  Percent 
Repairable  This  Station  may  all  drive  continuous  improvement 
at  a  lower,  internal  level.  Although  MICAP  and  TNMCS  rates 
do  not  directly  show  areas  for  continuous  improvement,  they 
can  lead  to  teamwork  between  supply  and  maintenance 
personnel,  and  they  inherently  motivate  supply  personnel  to 
continuously  improve. 

Ratings .  Nine  of  the  eighteen  indicators  had  a  sum 
median  and  sum  mean  of  10  or  less  out  of  a  possible  20. 
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Only  two  of  the  eighteen  indicators,  TNMCS  and  Warehouse 
Refusal  Rate,  were  rated  as  higher  than  15  in  both  sum 
median  and  mean.  All  of  the  five  focus  group  members  rated 
each  of  the  eighteen  indicators  on  the  two  criteria  of 
customer  focus  and  continuous  process  improvement.  This 
gave  a  total  of  180  individual  ratings  (5  individuals  x  18 
indicators  x  2  criteria) .  There  were  a  total  of  nineteen 
individual  ratings  of  9  or  10  (10.57  percent)  versus  forty- 
one  ratings  of  1  or  2  (22.78  percent) (see  Appendix  C) . 

Half  of  the  indicators  received  very  low  scores 
overall,  only  two  of  the  indicators  received  very  high 
scores  overall,  and  there  were  twice  as  many  individual 
ratings  of  1  or  2  versus  9  or  10.  This  implies  that  the 
currently  used  supply  management  indicators  are  very  weak  in 
terms  of  customer  focus  and  continuous  process  improvement. 

Additional  Findings.  There  was  general  consensus  among 
the  focus  group  that  there  is  a  need  for  measures  comparable 
between  different  types  of  bases,  different  commands,  and 
different  fleets.  Additionally,  some  measures  are  very 
useful  at  higher  organizational  levels,  but  not  very  useful 
at  lower  levels.  This  finding  is  consistent  with  management 
philosophies  wherein  strategic  goals  and  outlooks  are  more 
broad  in  focus  than  tactical  and  organizational  goals,  which 
are  progressively  narrower  in  focus.  MICAP  and  TNMCS  rates 
are  examples  of  this  finding.  They  both  give  commanders 
vital  information  on  weapon  system  status  but  neither  tell  a 
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supply  clerk  how  to  improve  the  supply  process  or  how  well 
the  clerk  is  directly  supporting  the  customer. 

Modification  of  Current  Indicators 

The  above  comments  were  used  in  conjunction  with  the 
findings  in  the  literature  research  to  modify,  and  in  some 
cases,  develop  new  indicators.  Some  indicators  were  given  a 
new  name.  These  new  indicators  are  designed  to  be  metrics, 
or  meaningful  indicators. 

Some  questions  which  arose  during  research  that  were 
considered  when  developing  the  new  indicators  were : 

1.  Does  supply  have  control  over  the  outcomes? 

2 .  Can  the  metric  be  used  to  compare  one  supply 
organization  to  another? 

3.  Can  time  and  cost  factors  be  incorporated? 

4.  Can  the  metric  induce  continuous  process  improvement 
and  show  the  level  of  customer/mission  support? 

As  noted  in  the  literature  review,  indicators  should  be 
controlled  by  the  function's  actions.  Without  supply  having 
control  over  the  outcome  of  the  indicators,  continuous 
process  improvement  cannot  occur.  Developing  metrics  to 
compare  supply  organizations  gives  a  baseline  for  comparison 
and  a  starting  point  for  continuous  process  improvement.  As 
recommended  by  Juran' s  quality  formula,  all  the  new  metrics 
are  in  terms  of  ratios  or  averages.  The  Daedalian  Award  can 
use  these  metrics  as  its  criteria.  As  supporting  elements 
of  customer  focus  and  continuous  process  improvement,  time 
and  cost  factors  were  applied  wherever  possible. 


62 


Analysis  of  Comments.  A  summary  of  comments  for 
improving  each  indicator  is  presented  in  Table  7  (all 
original  comments  are  presented  in  Appendix  B) . 


TABLE  7 

COMMENTS  FOR  IMPROVING  INDICATORS 


1.  Stockage  Effectiveness  (overall): 

-  need  to  look  at  stockage  effectiveness  by  weapon 
system,  not  overall. 

-  when  more  than  one  weapon  system  exists,  it  can 
create  conflicting  priorities. 

2.  Issue  Effectiveness  (overall): 

-  Should  be  measured  as  weapon  system  dependent. 

-  Potential  good  internal  measure.' 

-  Hope  the  customer  asks  for  the  right  thing. 

3.  Bench  Stock  Availability: 

-  measure  by  organization. 

-  survey  bench  stock  customers  for  satisfaction. 

4.  Stockage  Effectiveness  (primary  weapon  system): 

-  same  as  Stockage  Effectiveness  (overall) 

5.  Mission  Capable  Rate: 

-  can  be  broken  down  further  to  measure  true  supply 
support . 

6.  Total  Not  Mission  Capable  Supply: 

-  break  it  down  further  to  get  a  clearer  picture  of 
supply . 

7.  Recoverable/Consumable  Stockage  Effectiveness: 

-  too  detailed  to  be  useful. 

-  maybe  taking  the  stockage  effectiveness  down  to  a 
level  that  is  so  narrow  it  carries  little  weight. 

8.  Recoverable/Consumable  Issue  Effectiveness: 

-  similar  comments  as  #7  above. 

9.  Receipt-Not-Due-In  Rate: 

-  use  as  a  ratio  of  total  receipts. 

-  useful  as  a  rate  tracked  over  time. 
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10.  Percent  Repairable  This  Station  (overall): 

-  An  indicator  that  has  seen  it3  day  with  the 
introduction  of  2-level  maintenance. 

-  Under  2-level  maintenance,  this  is  becoming 
meaningless . 

11.  Inventory  Accuracy  (completed  plus  special): 

—  Great  indicator  (metric),  keep  as  is. 

12.  Releveling  Frequency: 

-  This  is  a  matter  of  policy. 

13.  Follow-Up  Frequency: 

-  Similar  comments  as  #12,  above. 

14.  Serviceable  Balance  With  No  Warehouse  Location: 

-  Excellent  lower  level  metric  for  internal  control. 

-  An  important  lower-tier  metric  supporting  inventory 
accuracy. 

15.  Item  Records  Past  Due  Inventory: 

-  Could  affect  the  customer,  but  usually  doesn't.  I/Rs 
frozen  for  inventory  more  likely  to  affect  customer. 

16.  System/General  Support  Division  Stockage  Effectiveness: 

-  Similar  comments  to  #7,  above. 

17.  System/General  Support  Division  Stockage  Effectiveness: 

-  Similar  comments  to  #8,  above. 

18.  Warehouse  Refusal  Rate: 

-  Keep  as  is . 

-  Very  important  leading  metric  of  inventory  accuracy. 

19.  General  Comments: 

-  I  believe  Reverse  Post  rates  are  one  of  the  most 
important  indicators  available  to  the  COS. 

-  Reverse  Post  rates  are  super  important! 

-  Let's  add  Reverse  Post  rates. 

-  Suggest  Reverse  Post  rates  be  included  as  an  internal 

_  supply  metric.  _ _ 


Tables  8  through  10  present  the  metrics  that  were 
developed  by  the  researchers.  The  rationale  for  the  actions 
taken  on  the  indicators  is  discussed  following  each  table. 
The  revised  indicators  are  presented  in  the  text  and  the 
formulas  for  these  metrics  are  listed  in  Appendix  D. 
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Unchanged  Indicators.  The  highly  rated  indicators  in 


Table  8  were  kept  in  their  original  forms. 


TABLE  8 

UNCHANGED  INDICATORS 


1.  Total  Not  Mission  Capable  Supply  Rate 

2.  Inventory  Accuracy  (completed  plus  special) 

3.  Warehouse  Refusal  Rate 


Total  Not  Mission  Capable  Supply  was  one  of  the  highest 
rated  indicators.  Further  breakdown  of  this  indicator  was 
accomplished  with  the  Mission  Capable  Rate,  below.  This 
indicator  was  kept  as  is:  TOTAL  NOT  MISSION  CAPABLE  SUPPLY. 

The  ratings  for  Inventory  Accuracy  were  relatively  high 
and  the  comments  were  extremely  positive.  In  addition, 
analysis  of  the  formula  did  not  indicate  any  areas  for 
improvement.  This  indicator  was  kept  as  is:  INVENTORY 
ACCURACY  (COMPLETED  PLUS  SPECIAL) . 

Warehouse  Refusal  Rate  received  the  highest  overall 
ratings.  All  of  the  comments  from  the  focus  group  were  very 
positive.  The  following  comment  was  typical  of  the  response 
from  the  focus  group. 

"Very  important  leading  metric  of  inventory  accuracy. 
Can,  if  managed  with  integrity,  approach  zero  over 
time.  In  the  vein  of  continuous  improvement,  when  the 
numbers  get  low,  they  can  usually  be  explained 
individually."  (Appendix  B) 

In  addition,  there  were  no  areas  for  improvement  of  the 

formula.  This  indicator  was  kept  as  is:  WAREHOUSE  REFUSAL 

RATE. 
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The  three  unchanged  indicators,  besides  being  strong  in 
both  customer  focus  and  continuous  process  improvement,  are 
very  useful  in  that  they  translate  well  between  the  various 
organizational  levels.  All  three  indicators  can  be  used  in 
various  forms  at  all  levels  from  the  individual  and  section 
level  to  the  Headquarters  and  MAJCOM  level.  An  example 
would  be  the  Warehouse  Refusal  Rate.  Specific  warehouse 
refusals  could  be  used  by  section  supervisors  to  identify 
training  requirements  for  individuals.  The  total  number  of 
refusals  could  be  used  by  mid— level  managers  to  identify 
training  effectiveness  and  reveal  problem  areas.  The 
MAJCOMs  can  use  Warehouse  Refusal  Rate  in  a  ratio  form  to 
compare  bases  within  their  command. 

Modified  Indicators.  The  indicators  presented  in  Table 
9  were  developed  by  modifying  some  of  the  original 
indicators . 


TABLE  9 

MODIFIED  INDICATORS 

1.  Stockage  Effectiveness  by  Weapon  System 

2.  Issue  Effectiveness  by  Weapon  System 

3.  Bench  Stock  Availability  by  Organization 

4.  Not  Mission  Capable  Supply  Rate 

5.  Receipt -Not-Due-In  Rate 

6.  Percent  Repairable  This  Station  (Three-level  Maintenance) 

7.  Serviceable  Balance  With  No  Warehouse  Location 


66 


Research  showed  that  base  supply  officers  have  very 
little  control  over  Stockage  Effectiveness.  Overall 
Stockage  Effectiveness  is  primarily  driven  by  funding. 
However,  the  focus  group' s  ratings  and  comments  indicated 
that  this  measure  may  still  be  important.  Tradeoffs  can  be 
made  between  weapon  systems.  Base  supply  officers  do  have 
some  control  over  this  tradeoff,  such  as  stocking  at  a 
higher  level  for  more  important  customers  and  missions. 

Based  on  the  analysis,  the  researchers  concluded  that 
STOCKAGE  EFFECTIVENESS  BY  WEAPON  SYSTEM  would  be  a  better 
measure . 

Because  Issue  Effectiveness  shows  how  well  demands  are 
anticipated,  it  is  potentially  a  very  good  measure. 
Monitoring  by  weapon  system  allows  the  supply  officer  to 
place  more  emphasis  on  the  most  important  customers.  This 
measure  was  modified  to  track  ISSUE  EFFECTIVENESS  BY  WEAPON 
SYSTEM. 

The  comments  indicated  Bench  Stock  Availability  would 
be  better  if  measured  by  organization,  as  some  organizations 
are  more  important  customers.  Therefore,  Bench  Stock 
Availability  was  modified  to  be  BENCH  STOCK  AVAILABILITY  BY 
ORGANIZATION. 

MICAP  rates  are  comprised  of  three  factors.  These 
factors  are  aircraft  or  weapon  systems  not-mission  capable 
because  of  maintenance,  supply,  or  both.  Because  total  not- 
mission  capable  supply  rates  include  only  the  supply  and 
both  factors,  these  researchers  eliminated  the  "both, " 
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leaving  NOT  MISSION  CAPABLE  SUPPLY,  which  replaced  the 
original  MICAP  rate. 

Based  on  the  comments,  Receipt-Not -Due-In  was  slightly 
modified  to  incorporate  Juran's  quality  ratio.  As  mentioned 
in  Chapter  3,  Juran's  quality  ratio  is  simply  applying 
ratios  to  measurements  in  order  to  establish  a  baseline  for 
continuous  process  improvement.  In  this  case  the  frequency 
of  deficiencies  equates  to  number  of  receipts  with  no 
corresponding  due— in  and  the  opportunities  for  deficiencies 
is  the  total  number  of  receipts.  The  new  indicator  is  still 
referred  to  as  RECEIPT-NOT-DUE-IN  RATE. 

Based  on  the  comments,  it  appeared  that  Percent 
Repairable  This  Station  had  lost  its  value.  However,  Col 
Morris,  HQ  AFMC/ALG  stated:  "Two-level  [maintenance]  only 
applies  to  avionics  and  engines  at  this  point,  and  not  all 
of  those.  There  will  still  be  a  lot  of  three-level 
maintenance  at  base  level  for  several  years."  A  slight 
modification  was  made  to  the  indicator  resulting  in  PERCENT 
REPAIRABLE  THIS  STATION  (3 -LEVEL  MAINTENANCE  ONLY) . 

The  comments  for  Serviceable  Balance  With  No  Warehouse 
Location  indicate  that  it  is  an  excellent  operational  level 
measure,  despite  the  mediocre  ratings.  In  an  attempt  to 
make  this  a  higher-level  indicator,  cost  factors  were  added. 
The  interim  measure  included  dollar  values  in  place  of 
"numbers  of  item  records,"  and  was  presented  as  SERVICEABLE 
BALANCE  WITH  NO  WAREHOUSE  LOCATION. 
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New  Indicators.  The  new  metrics  listed  in  Table  10 


were  developed  or  introduced  into  the  research.  Reverse 
Post  Rate  and  Supply  Cannibalization  Rate  are  indicators 
that  are  currently  used  in  varying  forms  but  did  not  meet 
the  criteria  for  the  most  commonly  used  supply  indicators 


TABLE  10 
NEW  INDICATORS 

—  ________________________  ________  — 

2 .  Not  Stocked  Backorder  Ratio 

3.  Bench  Stock  MICAP  Rate 

4 .  Item  Records  Frozen  For  Inventory  Rate 

5 .  Time  to  Clear  Frozen  Item  Records 

6.  Reverse  Post  Rate* 

7.  Unreported  Excess  Inventory  Ratio 

8.  Supply  Cann  Rate* 

*  =  denotes  currently  used  indicators  introduced  to  the 
research. 


In  addition  to  original  modification  made  to  Stockage 
Effectiveness  (overall) ,  cost  factors  in  terms  of  on-hand 
inventory  versus  authorized  inventory  were  incorporated  to 
develop  INVENTORY  EFFECTIVENESS  RATIO  (OVERALL) .  This 
indicator  is  calculated  by  multiplying  stockage 
effectiveness  by  the  percentage  of  authorized  inventory 
actually  on-hand.  An  organization  with  a  high  stockage 
effectiveness  rate  and  200  percent  of  authorized  inventory 
would  have  a  worse  Inventory  Effectiveness  Ratio  than 
another  organization  with  a  slightly  lower  stockage 
effectiveness  but  only  100  percent  of  authorized  inventory. 
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Using  this  indicator  would  continuously  drive  inventory 
reduction  and  provide  a  basis  for  a  fair  comparison. 

Based  on  comments  by  the  focus  group  which  emphasized 
the  fact  that  customers  don't  care  whether  the  item  is 
authorized  to  be  stocked,  a  new  measure  -  NOT  STOCKED 
BACKORDER  RATIO  was  extracted  from  the  issue  effectiveness 
indicators.  This  indicator  has  the  potential  to  lead  supply 
managers  to  be  proactive  in  forecasting  customer  demands . 

Bench  stock  assets  are  typically  not  critical  assets. 
However,  lack  of  bench  stock  assets  can  cause  aircraft 
groundings.  From  these  comments,  researchers  formulated  the 
BENCH  STOCK  MICAP  RATE.  This  indicator  is  the  ratio  of 
MICAP  hours  due  to  a  bench  stocked  item  divided  by  the  total 
number  of  MICAP  hours.  Although  this  rate  will  typically  be 
zero,  a  bench  stocked  item  causing  a  MICAP  is  a  very  strong 
indication  of  problems  in  the  supply  system  or 
miscommunication  between  supply  and  its  customers. 

Inventorying  in  base  supply  is  an  on-going  process. 
However,  non-routine  inventory  can  give  indications  of 
problem  areas.  Time  factors  which  relate  to  customer 
support  are  always  important  to  customers  and  were 
incorporated  in  this  indicator.  Based  on  comments  and 
analysis,  two  indicators  were  derived  from  Item  Records  Past 
Due  Inventory:  ITEM  RECORDS  FROZEN  FOR  INVENTORY  RATE 
(Number  of  Item  Records  Frozen  for  Non-routine  Inventory 
divided  by  Total  Item  Records)  and  TIME  TO  CLEAR  FROZEN  ITEM 
RECORDS  (Average  Time  to  Clear  Non-routine  Frozen  Item 


70 


Records).  The  first  indicator  was  developed  using  Juran' s 
quality  ratio  and  the  second  indicator  was  developed  by 
using  a  simple  average  for  the  baseline. 

Whenever  a  major  input  error  is  made  to  the  SBSS, 
another  transaction  (a  reverse  post)  must  be  created  to 
remove  the  error  from  the  system.  Although  Reverse  Post 
Rate  was  not  one  of  the  most  commonly  used  indicators,  it  is 
currently  being  tracked  by  Air  Education  and  Training 
Command.  As  indicated  in  the  general  comments,  reverse 
posts  are  a  direct  measure  of  the  efficiency  of  the  supply 
operation.  They  can  indicate  training  deficiencies,  system 
anomalies,  and  other  problems.  This  measure  translates  well 
between  the  various  levels  of  management.  This  indicator 
can  be  used  by  upper  level  management  as  an  indicator  of  the 
quality  of  training,  while  lower  level  management  can  use 
Reverse  Post  Rates  to  determine  specific  areas  in  need  of 
training.  For  these  reasons,  REVERSE  POST  RATE  was  added  to 
the  research. 

Based  on  a  literature  review  which  indicated  that  cost 
factors  are  heavily  emphasized  in  the  private  sector,  the 
researchers  developed  an  indicator  which  possesses  this 
factor.  Inventory  was  used  as  the  cost  factor  for 
UNREPORTED  EXCESS  INVENTORY  RATIO.  This  indicator  is 
calculated  by  dividing  the  total  dollar  value  of  excess 
materiel  by  the  total  dollar  value  of  the  inventory. 

The  sponsor  of  this  research  requested  that 
cannibalization  rates  be  incorporated  as  potential  metrics 
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to  be  reviewed  by  supply  officers.  Cannibalization  results 
from  the  customer  being  unable  to  wait  for  an  asset  from 
supply.  For  example,  a  part  is  unavailable  from  base  supply 
and  an  aircraft  needs  that  part  to  fly.  Maintenance  may 
pull  that  part  from  another  aircraft  and  put  it  on  the 
broken  aircraft.  Cannibalization  always  results  in  extra 
manhours  and  sometimes  results  in  extraordinary  cost  outlays 
in  the  form  of  damaged  aircraft  parts. 

Researchers  queried  several  aircraft  maintenance 
officers  and  concluded  that  there  was  no  set  formula  for 
deriving  cannibalization  rates.  Therefore,  researchers 
developed  a  measure  that  only  includes  cannibalization 
actions  attributable  to  supply  functions.  This  indicator  is 
SUPPLY  CANN  RATE. 

Deleted  Indicators.  Table  11  lists  original  indicators 
that  were  deleted  from  the  research.  The  rationale  for 
deletion  is  presented  in  the  text  following  the  table. 

Due  to  the  negative  comments,  low  ratings,  and  the  fact 
that  Stockage  Effectiveness  and  the  Issue  Effectiveness 
rates  were  already  modified,  Recoverable/Consumable  and 
SSD/GSD  Stockage  and  Issue  Effectiveness  rates  were  deleted. 
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TABLE  11 

DELETED  INDICATORS 


1.  Recoverable/Consumable  Stockage  Effectiveness 

2.  Recoverable/Consumable  Issue  Effectiveness 

3.  SSD/GSD  Stockage  Effectiveness 

4.  SSD/GSD  Issue  Effectiveness 

5.  Releveling  Frequency 

6.  Follow-up  Frequency 


Recoverable/Consumable  rates  break  down  stockage  and 
issue  effectiveness  by  type  of  assets.  Both  type  of  assets 
are  used  by  most  supply  customers .  Breaking  down  stockage 
and  issue  effectiveness  into  these  two  categories  does  not 
benefit  the  customers  nor  help  supply  managers  improve 
customer  support. 

System  Support  Division  and  General  Support  Division 
identify  sources  of  funding  for  assets.  Base  supply  has 
little  or  no  control  over  established  funding  levels  and, 
again,  this  information  does  not  benefit  the  customers  nor 
help  supply  managers  improve  customer  support. 

The  Releveling  Frequency  and  Follow-Up  Frequency  are 
not  controlled  by  base  supply  and  are  automatically 
accomplished  by  the  SBSS.  Improved  computer  technology  has 
made  these  two  measures  virtually  obsolete.  Because  base 
managers  have  no  control  over  these  indicators  and  the 
ratings  were  extremely  low,  these  indicators  were  deleted. 

The  above  procedure  resulted  in  a  list  of  18  indicators 
to  be  validated  by  a  second  focus  group  meeting.  These 
indicators  are  listed  in  Tables  8  through  10  and  Appendix  D. 
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Indicator  Verification.  The  18  revised  indicators  were 


presented  to  the  same  group  of  supply  officers  in  a  second 
focus  group  meeting.  The  officers  gave  comments  and  rated 
the  revised  indicators,  using  similar  procedures  as  before. 
No  verbal  discussion  was  necessary,  as  the  officers  were 
already  familiar  with  the  indicators  and  most  indicators 
were  modified  based  upon  their  comments.  Comments  and 
ratings  from  the  second  focus  group  meeting  are  listed  in 
Appendices  E  and  F.  The  ratings  are  summarized  in 
Table  12. 

Overall,  the  revised  indicators  were  rated  higher  than 
the  original  indicators .  While  ten  of  the  original 
indicators  had  combined  median  scores  of  less  than  11,  only 
three  revised  indicators  scored  that  low.  Eight  of  the 
revised  indicators  had  median  scores  of  greater  than  15, 
while  only  three  of  the  original  indicators  scored  as  well. 

The  indicator  "Serviceable  Balance  with  No  Warehouse 
Location"  was  modified  to  incorporate  the  dollar  value  of 
inventory  in  place  of  the  total  number  of  line  items.  This 
modification  proved  to  be  fruitless  because,  as  pointed  out 
by  the  focus  group,  one  high  dollar  value  item  or  many  low 
dollar  value  items  may  dramatically  skew  this  indicator. 
Upon  analysis  of  the  comments  from  both  focus  group 
meetings,  researchers  decided  to  retain  this  indicator  in 
its  original  form. 
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Median 

Mean 

18 

17.2 

18 

16.6 

16 

15.8 

16 

15.2 

15 

14.4 

15 

14.0 

15 

-.3.8 

15 

13.6 

14 

14.4 

14 

13.2 

14 

12.6 

12 

12.2 

12 

11.8 

12 

10.6 

11 

12.0 

11 

10.8 

9 

9.8 

9 

9.2 

TABLE  12 

RATINGS  OF  REVISED  INDICATORS 
INDICATOR 

* BENCH  STOCK-MI CAP  RATE 
BENCH  STOCK  AVAILABILITY  (BY  ORGANIZATION) 
*TIME  TO  CLEAR  FROZEN  ITEM  RECORDS 
SUPPLY  CANN  RATE  (BY  WEAPON  SYSTEM) 

STOCKAGE  EFFECTIVENESS  (EACH  WEAPON  SYSTEM) 
ISSUE  EFFECTIVENESS  -  (EACH  WEAPON  SYSTEM) 
NOT  MISSION  CAPABLE  SUPPLY  RATE 
WAREHOUSE  REFUSAL  RATE 
♦ITEM  RECORDS  FROZEN  FOR  INVENTORY  RATE 
REVERSE  POST  RATE 
TOTAL  NOT  MISSION  CAPABLE  SUPPLY 
♦NOT— STOCKED  BACKORDER  RATIO 
♦INVENTORY  EFFECTIVENESS  RATIO  -  (OVERALL) 
♦UNREPORTED  EXCESS  INVENTORY  RATIO 
PERCENT  REPAIRABLE  THIS  STATION  (3-LEVEL  MX) 
INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 
RECEIPT— NOT— DUE— IN  RATE 

SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOC 


*  =  denotes  indicators  developed  by  the  authors. 


A  t-test  using  median  ratings  was  used  to  determine 
whether  the  revised  indicators  were,  as  a  group, 
statistically  better  than  the  original  indicators.  At  a 
significance  level  of  .05,  this  test  determined  that  there 
was  sufficient  evidence  to  conclude  that  the  revised 
indicators  are  an  improvement  over  the  original  indicators. 
The  results  of  the  t-test  are  presented  in  Table  13. 
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TABLE  13 


T-TEST  RESULTS  -  FIRST  VS.  SECOND  FOCUS  GROUP 


Ml  M2  s  n  t  ta  R 

1C . 28  13.67  2.68  5  2.83  2.132  p  *  .02 


Indicator  Validation.  Because  the  revised  indicators 
were  statistically  better  than  the  original  indicators,  an 
attempt  was  made  to  validate  these  findings  across  the  Air 
Force.  Questionnaires  were  sent  to  all  field  grade  supply 
officers  at  Headquarters  Air  Combat  Command,  Air  Mobility 
Command,  Air  Education  and  Training  Command,  Space  Command, 
Air  Force  Logistics  Management  Agency,  and  Systems  Service 
Center.  Five  questionnaires  were  sent  to  field  grade  supply 
officers  at  other  offices  on  Wright-Patterson  AFB.  Thirty- 
four  questionnaires  were  sent  and  21  were  returned  for  a 
61.76  percent  return  rate,  which  is  comparable  with  typical 
return  rates  for  AFIT  surveys  sent  to  DOD  organizations 
(Shane,  1993) .  The  results  of  these  questionnaires  were 
used  to  determine  the  validity  of  this  research  for  all 
MAJCOMs  throughout  the  Air  Force.  The  questionnaire  can  be 
found  in  Appendix  G. 

Due  to  different  sizes  between  the  focus  group  and  the 
survey  group  (5  vs  21),  nonparametric  statistics  were  used 
to  compare  the  results.  The  researchers  ranked  the 
indicators  from  highest  to  lowest  median,  with  ties  being 
broken  in  favor  of  the  highest  mean  and,  if  necessary, 
lowest  standard  deviation  (see  Appendices  F  and  H) . 
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Because  the  average  median  from  the  Air  Force  sample 
was  over  two  points  higher  than  the  focus  group  (15.8  vs 
13,67),  a  one-tailed  Wilcoxon  Rank  Sum  Test,  using  a 
significance  level  of  .05,  was  conducted  to  determine  if  the 
Air  Force  sample  gave  higher  ratings  than  the  focus  group. 
The  result  of  this  test  is  shown  in  Table  14,  and  the  rank 
sums  are  presented  in  Table  15.  The  results  support  the 
conclusion  that  the  median  ratings  from  the  survey  group  are 
statistically  different  than  the  median  ratings  from  the 
focus  group. 


TABLE  14 

WILCOXON  RANK  SUM  TEST  -  ONE  TAILED 


3!f  51s  n,  jd 

409  257  18  270  p  «  .007 
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TABLE  15 

RANK  SUMS  FOR  THE  WILCOXON  RANK  SUM  TEST 


Focus 

Group 

Survey  Group 

Indicator  Median 

Rank 

Median 

Rank 

Unrpt  Excess 

12 

32 

14 

25 

Stock  Eff 

15 

16 

18 

4 

Inv  Eff  Ratio 

12 

31 

12 

30 

Issue  Eff 

15 

18 

16 

13 

NMCS  Rate 

15 

19 

19 

1 

TNMCS 

14 

26 

17 

7 

Rep— Not— D/ I 

9 

35 

14 

23 

%  RTS 

11 

33 

16 

15 

Inv  Acc 

11 

34 

17 

8 

Srv  Bal  W/no  Loc 

9 

36 

17 

9 

I/Rs  Frozen 

14 

22 

13 

28 

Time  to  Clr  I/Rs 

16 

11 

14 . 5 

21 

Rev  Post  Rate 

14 

24 

15 

17 

Whse  Ref  Rate 

15 

20 

16 

10 

Bnch  Stk  Avail 

18 

6 

18 

2 

Not-Stk  B/O 

12 

29 

14 

27 

Bnch  Stk  MICAP 

18 

3 

18 

5 

Supply  Cann 

16 

14 

16 

12 

Total  Rank  Sum: 

409 

257 

*  See  Appendix  D 

for  full  name  of 

indicators . 

Five  of  the  six  newly  developed  indicators  -  Inventory 
Effectiveness  Ratio,  Not  Stocked  Backorder  Ratio,  Item 
Records  Frozen  for  Inventory  Rate,  Time  to  Clear  Frozen  Item 
Records,  and  Unreported  Excess  Inventory  Ratio  -  were  rated 
in  the  bottom  six  by  the  survey  group.  The  comments  from 
the  survey  group  indicate  that  they  were  uncomfortable  with 
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indicators  with  which  they  were  not  familiar.  These  same 
indicators  ranked  3rd,  9th,  12th,  13th,  and  14th  in  the 
second  focus  group  ratings.  The  results  from  the  survey 
group  imply  that  the  supply  managers  rated  familiar 
indicators  higher  than  unfamiliar  indicators.  The  mixed 
ratings  from  the  second  focus  group  can  be  attributed  to  the 
opportunity  to  discuss  the  unfamiliar  indicators. 

The  sixth  newly  developed  indicator.  Bench  Stock  MICAP 
Rate  ranked  first  in  the  focus  group  and  fourth  in  the 
survey  group.  Even  though  this  is  a  newly  developed 
indicator,  the  two  components  of  Bench  Stock  and  MICAP  are 
indicators  that  are  currently  used  and  are  familiar  to 
supply  officers.  In  addition  to  familiarity,  this  indicator 
shows  customer  and  mission  support  as  well  as  indicating 
potentially  serious  problems  within  the  supply  system. 

Once  the  determination  was  made  that  the  Air  Force 
ratings  were  higher  than  the  focus  group  ratings,  the 
Spearman's  Rank  Correlation  Coefficient  was  calculated  to 
determine  if  the  focus  group  and  the  sample  rated  the 
metrics  in  a  similar  order.  The  results  of  this  test  are 
presented  in  Table  16.  R,  is  the  calculated  rank 
coefficient  and  R,.  is  the  table  value.  The  squared 
differences  are  listed  in  Table  17.  The  results  of  this 
test  support  the  conclusion  that  the  two  groups  rated  the 
indicators  in  a  similar  order. 
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TABLE  16 

SPEARMAN'S  RANK  CORRELATION  COEFFICIENT 


r. 

r* 

n 

P 

.3994 

.399 

18 

p  =  .05 

TABLE  17 

SQUARED  DIFFERENCES  FOR  SPEARMAN'S 

RANK  CORRELATION 

Indicator* 

Focus  Gp 
Rank 

Survey 

Rank 

Gp 

Diff 

Diff2 

Unrpt  Excess 

14 

15 

-1 

1 

Stock  Eff 

5 

3 

2 

4 

Inv  Eff  Ratio 

13 

18 

-5 

25 

Issue  Eff 

6 

10 

-4 

16 

NMCS  Rate 

7 

1 

6 

36 

TNMCS 

11 

5 

6 

36 

Rep— Not— D / I 

17 

14 

3 

9 

%  RTS 

15 

11 

4 

16 

Inv  Acc 

16 

6 

10 

100 

S  Bal  W/no  Loc 

18 

7 

11 

121 

I/Rs  Frozen 

9 

17 

-8 

64 

TTC  Fzn  I/Rs 

3 

13 

-10 

100 

Rev  Post  Rate 

10 

12 

-2 

4 

Whse  Ref  Rate 

8 

8 

0 

0 

Bnch  Stk  Avail 

2 

2 

0 

0 

Not-Stk  B/O 

12 

16 

-4 

16 

Bnch  Stk  MICAP 

1 

4 

-3 

9 

Supply  Cann 

4 

9 

-5 

25 

Sum  of  Diff2 

*  See  Appendix 

D  for  full 

name  of  indicators. 
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A  visual  examination  of  Table  17  reveals  that  four  of 


the  indicators  had  significantly  different  rankings.  Item 
Records  Frozen  for  Inventory  Rate  and  Time  to  Clear  Frozen 
Item  Records  were  rated  much  higher  by  the  focus  group. 

This  can  be  attributed  to  the  fact  that  these  two  indicators 
were  included  in  the  research  primarily  due  to  the  comments 
made  in  the  first  focus  group  meeting.  Pride  of  ownership 
and  the  opportunity  for  discussion  might  have  influenced  the 
higher  ratings. 

Serviceable  Balance  with  no  Warehouse  Location  was 
modified  to  reflect  dollar  values  when  presented  to  the 
second  focus  group.  The  original  form  of  this  indicator 
received  a  median  of  14,  whereas  the  modified  form  received 
a  median  of  11.  As  mentioned  earlier,  results  of  the 
discussion  indicated  that  this  modification  was  not 
appropriate.  Therefore,  the  original  version  was  submitted 
to  the  survey  group. 

Inventory  Accuracy  was  one  of  the  three  indicators  that 
was  not  changed  between  the  focus  group  meetings.  .  It  and 
the  other  two  unchanged  indicators,  TNMCS  Rate  and  Warehouse 
Refusal  Rate,  all  scored  lower  the  second  time  even  though 
the  average  median  increased.  This  implies  that  the  focus 
group  was  rating  the  indicators  relative  to  the  other 
indicators . 
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Summary 


Based  on  the  results  of  the  two  nonparametric  tests, 
the  Air  Force  sample  rated  the  revised  indicators  higher 
than  the  focus  group,  but  in  a  similar  order.  These 
findings  add  validity  to  the  research  and  show  that  these 
metrics  have  good  potential  for  Air  Force  application. 

The  relationship  between  the  18  indicators  and  the 
Quality-Based  Metrics  Framework  is  presented  in  Table  18. 
These  ratings  were  given  by  the  authors  based  on  their 
knowledge  of  the  indicators,  a  review  of  the  formulas  for 
the  indicators,  and  analyses  of  the  focus  group  and  survey 
comments.  An  "S"  implies  a  strong  presence,  a  "Wr  implies  a 
weak  presence,  and  a  indicates  no  presence  of  the 

characteristic  in  the  indicator. 

The  revised  indicators  were  significantly  better  than 
the  original  indicators.  Of  the  18  revised  indicators,  ten 
were  strong  in  both  customer  focus  and  continuous  process 
improvement  compared  to  only  two  of  the  original  indicators. 
The  other  eight  revised  indicators  were  strong  in  either 
customer  focus  or  continuous  process  improvement.  Ten  of 
the  original  indicators  were  strong  in  neither  customer 
focus  nor  continuous  improvement.  Time  factors  were 
incorporated  in  three  of  the  revised  indicators  versus  none 
of  the  original  indicators.  Cost  factors  were  incorporated 
in  13  of  the  revised  indicators  and  four  were  considered  to 
have  strong  presence.  Cost  factors  only  had  a  strong 
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presence  in  one  of  the  original  indicators.  Overall,  there 
were  34  "S"  ratings  out  of  72  possible  (47.22  percent). 


TABLE  18 


RELATIONSHIP  BETWEEN  REVISED  INDICATORS  AND  FRAMEWORK 


INDICATOR 

^CF 

Time 

Cost 

Unreported  Excess  Inv  Ratio 

- 

s 

- 

s 

Stockage  Effectiveness  (EWS) 

S 

s 

- 

w 

Inventory  Effectiveness  Ratio 

w 

s 

- 

s 

Issue  Effectiveness  (EWS) 

s 

s 

- 

w 

Not  Mission  Capable  Supply  Rate 

s 

s 

- 

- 

Total  Not  Mission  Capable  Supply 

s 

s 

- 

- 

Receipt-Not-Due-In  Rate 

w 

s 

- 

w 

%  Repairable  This  Station  (3  LM) 

s 

s 

- 

w 

Inventory  Accuracy 

w 

s 

- 

s 

Serv  Balance  w/No  Warehouse  Loc 

w 

s 

- 

w 

Item  Records  Frozen  for  Inv  Rate 

w 

s 

w 

s 

Time  to  Clear  Frozen  Item  Records 

s 

s 

s 

- 

Reverse  Post  Rate 

- 

s 

- 

- 

Warehouse  Refusal  Rate 

s 

s 

- 

w 

Bench  Stock  Avail  (by  org) 

s 

s 

- 

w 

Not-Stocked  Backorder  Ratio 

s 

s 

-  ' 

w 

Bench  Stock-MICAP  Rate 

s 

s 

w 

- 

Supply  Cann  Rate 

s 

s 

- 

w 

V.  Conclusions  and  Recommendations 


Overview 

This  chapter  discusses  the  overall  conclusions  based  on 
the  findings  and  analysis  of  the  investigative  questions 
presented  in  the  previous  chapters .  Recommendations  are 
then  made  for  Air  Force-wide  application  for  the  modified 
supply  metrics  and  potential  use  of  the  quality-based 
metrics  framework  throughout  the  logistics  community. 
Finally,  topics  are  suggested  for  future  research  in  the 
area  of  quality-based  metrics  in  logistics. 

Research  Conclusions 

Conclusion  #1.  Ratio  data  should  be  used  in  metrics 
whenever  possible.  Using  ratio  data  in  metrics  establishes 
a  baseline  for  comparison  between  diverse  organizations. 
There  may  be  a  great  disparity  between  the  workloads  of 
different  supply  organizations.  A  small  organization  may 
have  the  fewest  rejects  overall  but  have  the  highest  ratio 
of  rejects  to  inputs.  Without  using  ratios,  a  false  picture 
of  the  organizations  is  presented  to  the  higher  commands. 

This  baseline  can  also  be  used  internally  with  other 
quality  improvement  tools  to  strengthen  continuous  process 
improvement.  Tracking  the  total  number  of  rejects  from 
month  to  month  will  not  give  a  clear  indication  of 
efficiency  or  improvement.  If  the  number  of  rejects  doubles 
from  the  previous  month  but  the  total  number  of  transactions 
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triples,  then  this  is  an  improvement.  If  ratio  data  are  not 
used,  then  this  would  appear  to  be  a  retrogression.  Using 
ratio  data  would  also  allow  sections  to  apply  other  quality 
improvement  tools  such  as  statistical  process  control. 

Conclusion  #2.  There  is  very  little  consensus  among 
the  Air  Staff  and  various  MAJCOMs  as  to  which  indicators  are 
important.  Only  one  indicator  out  of  28  (Bench  Stock 
Availability)  was  identified  by  all  sources.  Out  of  the  14 
indicators  identified  by  AFR  900-14,  only  two  (Inventory 
Accuracy  and  Bench  Stock  Availability)  are  being  reviewed  by 
at  least  one  MAJCOM.  Only  nine  of  the  28  indicators  are 
reviewed  by  more  than  one  MAJCOM.  Overall,  only  ten  of  the 
28  indicators  (36%)  were  identified  by  more  than  one  source. 

MAJCOMs  may  not  be  emphasiring  the  indicators  found  in 
AFR  900-14  because  they  are  already  being  reviewed  by  Air 
Staff.  However,  if  an  indicator  is  important  to  the  higher 
echelons,  it  should  be  important  to  the  lower  echelons  as 
well.  Although  the  actual  missions  of  the  MAJCOMs  vary, 
they  are  all  supported  by  the  SBSS.  Therefore,  there  should 
be  more  consensus  among  the  different  MAJCOMs  as  to  which 
indicators  are  important. 

Conclusion  #3.  Indicators  at  all  levels  should  support 
the  strategic  goals  of  the  organization.  Different 
indicators  are  appropriate  at  various  organizational  levels 
because  each  organizational  level  requires  unique  management 
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information.  Even  though  most  of  the  data  is  derived  from 
the  same  source,  different  levels  within  the  same 
organization  usually  apply  the  data  differently.  This 
appears  to  correspond  with  the  management  philosophies 
wherein  strategic  goals  require  more  general  information 
while  tactical  and  operational  goals  require  progressively 
more  specific  information  (Dixon  and  others,  1990:166). 
This  conclusion  is  depicted  in  Figure  3. 


Figure  3.  Quality-Based  Metrics  Framework 


The  figure  above  represents  a  framework  on  which 
quality-based  metrics  should  be  built.  Time  and  cost  are 
underlying  supporting  elements  for  continuous  process 
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improvement  and  customer  focus.  Continuous  process 
improvement  and  customer  focus  are  the  major  components  of 
quality-based  indicators.  Indicators  exist  for  all 
organizational  levels.  Metrics  are  a  subset  of  all 
management  indicators .  Metrics  are  meaningful  indicators 
that  allow  managers  to  take  action  and  result  in  process 
improvement . 

Operational  goals  should  support  the  tactical  and 
strategic  goals  of  the  organization.  Metrics  should  be 
present  at  every  level  to  link  all  performance  for  the 
attainment  of  strategic  goals  (Hall  and  others,  1991:77; 
Dixon  and  others,  1990:72;  Williams,  1982:458-461;  Szilagyi 
and  Wallace,  1980:336-337) . 

Conclusion  #4.  The  characteristics  o '  customer  focus 
and  continuous  process  improvement  can  be  applied  to  improve 
current  indicators  in  other  logistics  areas.  By  applying 
the  quality-based  metrics  framework  developed  through  the 
literature  review,  performing  content  analysis  of  the 
comments  made  by  the  focus  group,  and  applying  ratios 
wherever  appropriate,  the  original  indicators  were  modified 
to  bring  them  more  in-line  with  the  TQM  philosophy. 
Improvements  were  verified  by  having  the  same  focus  group 
rate  the  revised  group  of  indicators.  The  revised 
indicators  were  significantly  better  than  the  original 
indicators.  The  results  were  validated  by  surveying  field 
grade  supply  officers  throughout  the  Air  Force.  The 
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surveyed  officers  rated  the  indicators  significantly  higher 
than  the  focus  group,  implying  potential  Air  Force-wide 
application  of  the  quality-based  supply  metrics. 

Although  the  research  applied  this  framework  directly 
to  supply  indicators,  the  literature  review  indicates  that 
this  framework  is  appropriate  across  the  logistics  spectrum. 
Logistics  managers  at  the  different  organizational  levels 
(see  Figure  3)  can  apply  this  framework  to  their  currently 
used  indicators  to  bring  them  more  in-line  with  the  TQM 
philosophy. 

In  a  contracting  organization,  a  quality-based 
indicator  might  be  a  ratio  of  incorrectly  processed  purchase 
requests  to  total  number  of  requests.  A  transportation 
metric  could  be  the  percentage  of  requests  for  vehicles 
satisfied.  Services  squadrons  could  measure  customer 
satisfaction  at  the  various  dining  facilities.  The  Quality- 
Based  Metrics  Framework  is  readily  evident  in  many 
maintenance  indicators .  An  example  of  a  maintenance  metric 
is  TNMCM  Rate,  which  is  equivalent  to  the  TNMCS  Rate.  These 
metrics  directly  relate  to  customer  focus  and  can  be  used  to 
drive  continuous  improvement . 

Conclusion  #5.  Many  Air  Force  supply  indicators  are 
unnecessary  as  they  do  not  add  value  to  the  supply  process 
or  are  not  linked  to  strategic  goals.  The  original  focus 
group  meeting  determined  that  the  currently  used  supply 
indicators,  as  a  group,  did  not  possess  the  characteristics 
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of  the  quality-based  metrics  framework.  This  was  reflected 
in  the  ratings  given  to  the  indicators  by  the  focus  group. 
The  group  rated  9  of  the  18  indicators  as  having  a  sum 
median  and  sum  mean  of  10  or  less  out  of  a  possible  20. 

Only  3  of  the  18  indicators  had  sum  means  of  higher  than  15. 
On  the  two  criteria  (customer  focus  and  continuous  process 
improvement)  for  all  18  indicators,  there  were  a  total  of  19 
individual  ratings  of  9  or  10  versus  41  ratings  of  1  or  2 
(see  Appendix  C) . 

The  authors  deleted  6  of  the  18  original  indicators 
(33%) .  These  indicators,  listed  in  Table  11,  did  not  have 
the  necessary  characteristics  required  for  metrics.  As 
mentioned  in  the  literature  review,  the  Wang  Corporation 
deleted  40%  of  their  indicators  when  going  through  a  similar 
process . 

Recommendations 

Due  to  the  sources  of  information  on  the  most  commonly 
used  indicators,  the  indicators  presented  throughout  this 
research  are  primarily  strategic  in  nature.  The  revised 
indicators  have  numerous  potential  direct  applications. 

Recommendation  #1 .  The  researchers  recommend  that  the 
revised  metrics  (see  Appendix  D)  be  used  to  replace  those 
measures  used  in  the  evaluation  of  the  Air  Force  Daedalian 
and  Supply  Effectiveness  awards.  Of  the  fourteen  indicators 
used  as  Daedalian  Award  criteria,  ten  were  rated  by  the 


89 


focus  group  as  having  a  sum  median  rating  of  less  than  10. 
Only  two  of  the  modified  indicators  rated  by  the  same 
officers  had  a  sum  median  rating  of  less  than  10;  both  had 
sum  median  ratings  of  9.  The  current  measures  used  in  the 
evaluation  for  these  awards  favor  smaller  organizations  with 
easy-to-maintain  weapon  systems.  The  revised  metrics 
establish  a  fair  baseline  to  compare  diverse  organizations. 

Recommendation  #2.  Supply  indicators  should  be  revised 

and  incorporated  into  the  M-32.  Although  the  main  focus  of 

this  research  was  on  strategic  measures,  the  focus  group 

comments  as  well  as  the  comments  from  the  surveyed  officers 

indicate  that  the  framework  of  customer  focus  and  continuous 

process  improvement  can  be  applied  to  metrics  at  different 

levels  of  management  within  supply.  The  following  comments 

from  the  survey  support  this  recommendation. 

"Your  metrics  can  all  be  rolled  up  to  the  MAJCOMs . 

I  propose  there  are  others  [metrics]  relevant 

[only]  at  the  wing  level."  (Lt  Col  Johnson,  ACC,  1993) 

"Good  metrics  which  inspire  the  right  actions  tell 
our  people  at  all  working  levels  where  we  want  to 
go  and  allow  us  to  turn  our  people  loose  to  get 
there  in  their  own  way."  (Col  Friel,  AFMC,  1993) 

A  review  of  all  current  supply  indicators  should  be 

conducted.  Supply  managers  should  apply  the  quality-based 

metrics  framework  to  revise  their  current  indicators.  All 

indicators  that  cannot  be  linked  to  strategic  goals  or  do 

not  add  value  to  the  supply  process  should  be  eliminated. 
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The  resulting  metrics  should  be  incorporated  into  the  M-32 
to  make  it  a  more  useful  management  tool. 

Recommendation  #3.  In  order  to  foster  continuous 
improvement  throughout  the  Air  Force,  the  Directorate  of 
Supply  at  Air  Staff  should  publish  a  periodic  newsletter 
highlighting  success  stories  in  quality  improvement.  The 

newsletter  could  contain  metrics  being  used  by  each  command 
and  ways  that  the  various  bases  are  applying  these  metrics 
for  continuous  improvement.  Accompanying  articles  could 
explain  how  specific  metrics  support  the  attainment  of 
strategic  goals,  which  will  educate  the  workers  about  the 
relevance  of  their  jobs  and  make  them  feel  more  responsible. 
This  newsletter  could  be  used  as  a  tool  for  benchmarking  by 
spreading  information  throughout  the  supply  community. 

Recommendation  #4.  The  authors  strongly  recommend  that 
Bench  Stock  MICAP  Rate  be  tracked  at  all  supply  units 
throughout  the  Air  Force.  This  metric  directly  links  supply 
performance  to  customer  satisfaction  and  mission 
accomplishment.  Bench  stock  items  are  relatively 
inexpensive  assets  that  the  customers  continually  use. 
Therefore,  there  is  seldom  an  excuse  for  supply  not  having 
these  assets  available.  If  the  lack  of  a  bench  stock  item 
causes  a  MICAP,  an  aircraft  is  grounded  due  to  poor  supply 
support . 
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Bench  Stock  MICAP  Rate  falls  in  the  category  of  "new" 
indicators  and  had  the  second  highest  overall  rating  between 
the  focus  group  and  the  survey  group.  It  was  rated  the 
highest  by  the  focus  group  and  second  highest  by  the  survey 
group.  This  result  is  significant,  considering  that  the 
other  five  "new"  indicators  were  rated  in  the  bottom  six  by 
the  survey  group.  The  following  summary  from  the  second 
focus  group  meeting  illustrates  the  usefulness  of  this 
indicator: 


Too  many  times  close  to  fiscal  year  end,  we 
cut  back  on  bench  stock  fill  to  control 
expenditures.  This  would  give  all  concerned  a 
measure  that  could  be  used  to  beef  up  stock  on 
certain  bench  stock  items  to  preclude  the  costly 
MICAP s. 

This  would  be  a  good  way  to  zero  in  on 
problems  caused  by  small  items.  For  want  of  a 
bench  stock  item,  the  mission  wasn't  flown.  We 
should  not  let  small  dollar  value  items  ground 
systems  or  aircraft.  This  should  be  very  useful 
in  showing  how  the  bench  stock  items  are  affecting 
mission  support.  (Appendix  E) 


Recommendation  #5 .  The  revised  metrics  should  be 
implemented  at  all  MAJCOMs.  Major  commands  can  also  use 
these  metrics  to  compare  diverse  subordinate  supply 
accounts.  Even  within  the  same  MAJCOM,  wings  with  similar 
weapon  systems  may  vary  in  size.  The  current  indicators 
favor  supply  organizations  that  support  smaller  wings.  By 
applying  ratios  to  most  of  the  indicators,  a  common  baseline 
can  be  developed  that  is  fair  to  different  organizations 
regardless  of  resources  available. 
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Recommendation  #6.  The  revised  metrics  should  be 
applied  at  all  base  supply  organizations.  Since  metrics 
drive  appropriate  actions.  Chiefs  of  Supply  could 
continuously  improve  by  benchmarking  those  organizations 
that  have  the  best  ratings  in  these  metrics.  The  previously 
recommended  newsletter  would  help  make  this  possible.  There 
are  numerous  other  direct  applications  of  these  metrics  at 
base  level.  Supply  organizations  could  apply  ratio  data,  in 
the  forms  of  Receipt-Not-Due-In  Rate,  Item  Records  Frozen 
for  Inventory  Rate,  Reverse  Post  Rates,  and  Bench  Stock 
MICAP  Rates,  to  implement  continuous  improvement  at  base 
level . 

Metrics  can  be  used  by  managers  throughout  base  supply 

to  empower  workers.  Metrics  that  are  linked  to  the 

strategic  goals  allow  all  workers  to  influence  the 

achievement  of  these  strategic  goals.  Once  workers  are 

educated  on  how  their  jobs  relate  to  the  strategic  goals, 

they  become  motivated  to  continuously  improve.  The 

relationship  between  metrics  and  empowerment  can  best  be 

explained  by  the  following  quote: 

"Good  metrics  are  one  of  the  most,  if  not  the  most, 
effective  ways  to  empower  our  people.  If  we  do  a  good 
job  of  empowering  our  people,  we  take  a  big  step 
forward  in  our  attempts  to  create  an  environment  of 
continuous  improvement."  (Col  Friel,  AFMC,  1993) 

Recommendation  #7.  The  authors  recommend  that  a  system 
be  developed  to  track  asset  flow  through  the  various 
segments  of  the  logistics  pipeline.  The  flow  of  assets 
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through  the  Air  Force  logistics  pipeline  has  been  thoroughly 
studied.  Computer  technology  is  available  to  track  assets 
through  the  entire  pipeline.  Successful  commercial 
companies  such  as  UPS  and  Federal  Express  can  locate  an 
asset  anywhere  in  the  pipeline  within  minutes.  The  authors 
had  difficulty  incorporating  time  factors  into  the  metrics. 
This  was  primarily  due  to  the  fact  that  customer  waiting 
times  are  not  tracked  by  the  supply  system.  Customer  wait 
times  can  only  be  tracked  manually. 

The  only  waiting  time  tracked  by  the  SBSS  is  the  amount 
of  time  it  takes  for  base  supply  to  receive  their  assets 
from  sources  of  supply.  It  does  not  take  into  consideration 
whether  the  assets  are  required  immediately  by  customers  or 
are  for  replenishment  of  stock.  The  ability  to  track  MI CAP 
assets  from  source  to  destination  would  greatly  improve 
customer  and  mission  support.  By  being  able  to  track  the 
individual  items  through  the  segments,  logistics  managers 
could  isolate  problem  areas  and  attempt  to  improve  those 
areas . 

Recommendation  #8.  Implement  a  retail  sales  survey. 
Using  the  list  of  customers  who  shop  at  the  Base  Service 
Store,  the  Tool  Issue  Center,  or  the  Individual  Equipment 
Section,  the  Retail  Sales  Section  of  base  supply  should  send 
out  a  monthly  customer  satisfaction  survey.  The 
questionnaire  should  contain  questions  which  reflect  levels 
of  customer  support,  with  a  score  of  1  implying  poor 
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customer  service  and  a  score  of  5  indicating  excellent 
customer  service.  This  information  could  be  fed  back  to  the 
workers  to  drive  continuous  improvement  and  improve  customer 
support . 

Suggestions  for  Further  Research 

This  research  uncovered  areas  that  should  be 
investigated  further.  Although  Total  Quality  Management  is 
not  a  new  philosophy,  application  of  TQM  in  the  area  of 
logistics,  and  especially  in  logistics  metrics,  is  still  in 
its  infancy.  Developing  metrics  that  support  TQM  requires 
looking  at  logistics  organizations  from  a  different 
perspective  than  traditional  management  philosophies. 

Suggestion  #1.  Research  should  be  conducted  to 
develop,  verify,  and  validate  metrics  in  other  logistics 
areas.  The  literature  review  indicates  that  the  Quality- 
Based  Metrics  Framework  developed  in  this  research  can 
translate  to  other  logistics  functions.  Therefore,  similar 
studies  should  be  conducted  using  focus  groups  to  apply  this 
framework  and  develop  metrics  for  Transportation, 
Maintenance,  Logistics  Plans,  Contracting,  and  Services.  A 
stronger  validation  could  be  conducted  by  incorporating 
ratings  and  comments  from  overseas  MAJCOMs . 

Suggestion  #2.  Develop  metrics  for  base  supply. 

Current  supply  indicators  at  all  organizational  levels 
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(operational,  tactical,  and  strategic)  should  be  examined 
and  the  Quality-Based  Metrics  Framework  applied  to  develop 
distinctive  metrics  for  each  of  the  respective  levels. 
Metrics  could  be  developed  for  each  section  within  the 
supply  organization  to  link  lower-level  performance  with 
strategic  goals.  The  revised  indicators  developed  in  this 
research  could  be  broken  down  into  useful  metrics  at  the  COS 
and  individual  section  levels.  An  example  is  Warehouse 
Refusal  Rate.  The  total  number  of  warehouse  refusals  would 
be  useful  to  the  COS  for  indication  of  training 
effectiveness,  while  individual  warehouse  refusals  by 
location  would  be  useful  to  the  section  supervisor  to 
identify  specific  individuals  requiring  training. 

Suggestion  #3.  Conduct  a  case  study  concerning 
implementation  of  the  revised  metrics.  This  study  could 
apply  revised  metrics  to  several  supply  organizations  in  the 
various  MAJCOMs  and  track  their  performance  over  a  period  of 
time,  verifying  whether  the  metrics  drive  continuous 
improvement  in  an  actual  work  environment.  This  study  could 
result  in  a  revised  set  of  metrics  and  also  determine  which 
metrics  work  and  which  do  not.  This  study  could  also 
investigate  how  the  metrics  help  empower  the  workers  and 
drive  the  appropriate  behavior. 
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Summary 

This  research  developed  the  Quality-Based  Metrics 
Framework,  which  consists  of  customer  focus  and  continuous 
process  improvement  with  the  underlying  supporting  elements 
of  time  and  cost.  A  review  of  pertinent  Air  Force 
publications  and  interviews  with  supply  analysts  at  CONUS 
MAJCOMs  identified  the  most  commonly  used  supply  indicators. 
This  list  contains  all  indicators  included  in  the  Daedalian/ 
Supply  Effectiveness  Award  criteria  and  other  indicators 
regularly  reviewed  by  more  than  one  CONUS  MAJCOM.  These 
indicators  were  found  to  be  of  little  use  to  base-level 
supply  officers.  The  original  indicators  were  then  modified 
to  bring  them  more  in-line  with  the  TQM  philosophy. 

Supply  management  indicators  have  remained  unchanged 
since  the  implementation  of  TQM  and  metrics  in  the  Air 
Force.  Metrics  are  one  of  the  most  important  tools  that  a 
manager  has  to  improve  the  quality  of  the  organization. 
Metrics  drive  process  improvement,  link  lower  level 
performance  to  strategic  goals,  and  can  even  empower 
workers.  Proper  application  of  revised  metrics  throughout 
the  Air  Force  and  the  Department  of  Defense  can  dramatically 
improve  the  achievement  of  national  defense  goals. 

Air  Force  Materiel  Command  (AFMC)  is  a  leader  in  the 
implementation  of  metrics  in  the  Air  Force.  The  command's 
sponsorship  and  application  of  this  research  reflects  the 
importance  of  metrics.  Customer  focus  and  continuous 
process  improvement  directly  relate  to  the  first,  second, 
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fourth,  and  fifth  AFMC  goals  of  (1)  satisfying  customer 
needs  in  war  and  peace;  (2)  enabling  people  to  excel;  (3) 
sustaining  technological  superio__;y;  (4)  enhancing  the 
excellence  of  business  practices;  and  (5)  operating  quality 
installations  (Hinneburg,  AFMC,  1991).  Implementation  of 
these  metrics  is  a  way  to  assess  achievement  of  AFMC's 
strategic  goals. 

If  the  Air  Force  supply  community  is  to  be  successful 
in  realizing  the  highest  possible  degree  of  quality 
improvement,  supply  indicators  must  be  geared  towards 
continuous  quality  improvement  rather  than  just  meeting  the 
minimum  goals.  As  General  McPeak  stated  in  his  video  "Two 
Kinds  of  Change, "  the  style  of  the  new  Air  Force  is  Quality 
Air  Force  and  our  credo  is:  "A  leadership  commitment  and 
operating  style  that  inspires  trust,  teamwork  and  continuous 
improvement. . . " (McPeak,  1992) . 
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Appendix  A:  Supply  Management  Indicators 
GENERAL  INFORMATION 

The  supply  management  indicators  presented  to  the  focus 
group  are  listed  below  along  with  the  formulas  used  to 
equate  the  indicators .  The  formulas  were  found  in  AFM  67-1 
and  AFR  900-14  and  associated  documents. 


1  -  3TOCKAGE  EFFECTIVENESS  -  (OVERALL) 


_ No.  Units  Issued _ 

No.  Units  Issued  +  No.  Units  B/O  -  No.  Units  B/o  4W 


2  -  ISSUE  EFFECTIVENESS  -  (OVERALL) 

_ No.  Units  Issued _ 

No.  Units  Issued  +  No.  Units  B/O 

3  -  BENCH  STOCK  AVAILABILITY 

Total  Line  Items  Authorized  -  Line  Items  Due-out 
Total  Line  Items  Authorized 


4  -  STOCKAGE  EFFECTIVENESS  -  (PRIMARY  WEAPON  SYSTEM) 

_ No.  PWS  Units  Issued _ 

No.  PWS  Units  Issued  +  No.  PWS  Units  B/O  -  No.  PWS  Units  B/O  - 


5  -  MISSION  CAPABLE  RATE 

NMCM  +  NMCS  +  NMCB  Hours 
Total  Possessed  Hours 

6  -  TOTAL  NOT  MISSION  CAPABLE  SUPPLY 

NMCS  +  NMCB  Hours 
Total  Possessed  Hours 
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7  -  RECOVERABLE/CONSUMABLE  STOCKAGE  EFFECTIVENESS 


_ _ Rec  Units  Issued _ 

Rec  Units  Issued  +  Rec  Units  B/O  -  Rec  Units  B/O 


_ Cons  Units  Issued _ 

Cons  Units  Issued  +  Cons  Units  B/O  -  Cons  Units  B/O  4 W 


8  -  RECOVERABLE /CONSUMABLE  ISSUE  EFFECTIVENESS 

_ Rec  Units  Issued _ 

Rec  ws  Units  Issued  +  Rec  WS  Units  B/O 


_ Cons  Units  Issued _ 

Cons  WS  Units  Issued  +  Cons  ws  Units  B/O 


9  -  RECEIPT— NOT— DUE— IN  RATE 

Total  Shipments  Received  With  No  Cor  responding  Due- in 

10  -PERCENT  REPAIRABLE  THIS  STATION  (OVERALL) 

Number  of  Units  Repairable  This  Station 
No .  Uni  ts  RTS  +  No .  Uni  ts  NRTS  +  No .  Uni  ts  Cond 


11  -  INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 

1-  Total  Units  Over  +  Short 
Total  Record  Balance 


12  -  RELEVELING  FREQUENCY 

No.  Completed  with  Date  of  Last  Transaction 
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13  -  FOLLOW-UP  FREQUENCY 

No.  Completed  with  Date  of  Last  Transaction 

14  -  SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 

Numer  of  Item  Records  With  Serv  Bal  and  No  Whse  Location 
Total  Number  of  Item  Records 


15  -  ITEM  RECORDS  PAST  DUE  INVENTORY 

Total  Item  Records  Past  Due  Inventory 
Total  Item  Records 


16  -  SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  &  GSD)  STOCKAGE 
EFFECTIVENESS 


_ No.  SSD  Units  Issued _ 

No.  SSD  Units  Iss  +  No.  SSD  Units  B/O  -  No.  SSD  Units  B/o  4 W 


_ No.  GSD  Units  issued _ 

No.  GSD  Units  Iss  +  No.  GSD  Units  B/O  -  No.  GSD  Units  B/o  4 W 


17  -  SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  &  GSD)  ISSUE 
EFFECTIVENESS 


_ No.  SSD  Units  Issued _ 

No.  SSD  Units  Issued  +  No.  SSD  Units  B/O- 


_ No.  GSD  Units  Issued _ 

No.  GSD  Units  Issued  +  No.  GSD  Units  B/O- 


18  -  WAREHOUSE  REFUSAL  RATE 


Total  Number  of  Warehouse  Refusals 
Total  Number  of  Issues 


Appendix  B:  Comments  From  First  Focus  Group 


1.  STOCKAGE  EFFECTIVENESS  -  (OVERALL) 


Need  to  track  trends  over  a  two  year  period.  An  effective 
metric  of  how  well  we  keep  what  we' re  authorized  on  the 
shelf.  Doesn't  account  for  those  things  our  customer  asks 
for  that  we  don't  have  levels  for.  Those  are  important,  too. 

I  like  this  indicator,  but  you  have  to  have  a  good  feel  for 
what  you  have  and  don't  have. 

This  indicator  is  nice  to  know,  however  base  level  managers 
have  little  control. 

A  primary  evaluation  tool  of  any  base  supply  organization. 
Dependent  on  depot  support  -  creates  lots  of  manual  work  for 
supply  personnel  as  part  of  follow-up. 

If  we  were  truly  responsive  and  customer  oriented,  we  would 
refuse  to  make  ourselves  better  by  excluding  the  4W  B/Os. 
While  we  don't  have  crystal  balls  to  project  what  the 
customer  wants  and  needs,  the  current  formula  allows  us  to 
artificially  inflate  how  well  we're  doing. 


2.  ISSUE  EFFECTIVENESS  -  (OVERALL) 


If  it  isn't  stocked,  you  can't  win  with  this  one. 

This  tells  the  customer  what  he  needs  to  know  -  we  ought  to 
be  more  interested  than  we  have  been.  Using  only  stockage 
effectiveness  lets  us  hide  behind  the  "we  aren't  authorized 
this"  syndrome.  The  customer  doesn't  care  whether  we  are 
auth  a  level  or  not.  He  needs  his  stuff. 

My  feeling  is  this  is  worthless.  Too  easy  to  manipulate. 
This  is  currently  being  misused  by  COSs  who  push  "numbers" 
in  an  effort  to  make  their  accounts  more  competitive. 

This  can  provide  you  with  information  regarding  how  well  you 
anticipate  customer  requirements.  Since  the  supply  system 
is  reactive  to  demands  rather  than  proactive,  this  indicator 
should  not  be  used  to  judge  effectiveness. 

Hope  the  customer  ask  for  the  right  thing.  Doesn't  keep  up 
with  the  rapid  changes  that  customers  are  experiencing. 

Again,  this  should  be  measured  as  weapon  system  dependent. 
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3.  BENCH  STOCK  AVAILABILITY 


Unfortunately,  we  tend  to  hide  behind  Computer  generated 
authorizations  in  lieu  of  what  the  customer  really  needs  on 
his  account.  Even  if  we  do  add  to  the  B/S  account  often  we 
don't  stock  adequate  quantities  in  supply  to  ensure  timely 
fill  of  depleted  B/S. 

Agree  this  is  important  to  the  customer.  Must  be  used  to 
drive  follow-up  behavior  in  stock  control.  My  experience  is 
that  customers  complain  more  quickly  about  bench  stock  than 
anything  else  if  support  isn't  good.  Must  mean  they  think 
it  is  important. 

Maintenance  tends  to  put  many  things  on  Bench  Stock  that 
aren't  really  needed,  and  may  not  be  used  much.  Just 
because  we  have  a  lot  of  those  things  in  stock  doesn't  tell 
us  how  well  we  are  supporting  the  customer.  If  we  measured 
this  BY  ORG,  we  would  probably  learn  that  this  measure  is 
more  valid  for  some  orgs  than  others  because  some  have  what 
they  SHOULD  have  on  bench  stock,  and  others  don't. 

Great  indicator  for  your  Bench  Stock  supervisor.  However, 
the  senior  supply  managers  don't  need  this  indicator.  The 
COS  might  want  to  survey  bench  stock  customers  for  service 
satisfaction  ratings. 

Good  indicator  for  supply  because  its  always  high.  Bench 
stock  still  confusing  to  many  customers.  Manually 
manipulated  near  end  of  year  to  spend  or  save  money. 


4.  STOCKAGE  EFFECTIVENESS  (PRIMARY  WEAPON  SYSTEM) 


Important  measure  of  level  of  support.  Fiscal  constraints 
prevent  continuous  improvement  past  a  certain  point.  Also, 
the  leveling  formula  limits  stockage  past  a  certain  level. 

We  measure  ourselves  on  this  one.  Our  customers  measure  us 
more  on  issue  effectiveness. 

Agree  this  is  important  measure.  However,  one  comment 
indicated  that  the  customer  is  more  concerned  with  Issue 
Effectiveness.  May  be  true,  but  I  believe  this  is  an 
invalid  measure  unless  it  is  used  internally  by  managers  who 
not  only  know  what  they  are  doing,  but  are  unwilling  to  lie 
to  themselves. 

Good  indicator  when  only  one  weapon  system  existed  at  a 
base.  When  more  than  one  exists,  creates  conflicting 
priorities . 
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Maybe  there  should  be  stockage  effectiveness  levels  for  OLD 
and  NEW  weapons  systems  at  any  given  installation.  A  COS 
who  has  had  a  system  loaded  for  10  years  should  show  a 
better  rate  that  if  s/he  had  a  year  old  system.  Perhaps  4Ws 
would  be  useful  for  NEW  systems  vice  OLD. 

Need  to  look  at  Stockage  Effectiveness  BY  WEAPON  SYSTEM,  not 
overall,  if  there  is  more  than  one  primary  weapon  system  on 
a  base.  Particularly  true  for  objective  wings  which  support 
numerous  weapon  systems. 

If  authorized  to  stock  item,  and  sufficient  funding  is 
available,  and  if  depot  managers  are  keeping  stocks  on  hand, 
and  if  contracting  folks  don't  drop  the  ball  in  the 
acquisition  process,  etc. . .  Supply  managers  at  base  level 
have  little  control .. .Good  feeder  information  for  top  level 
policy  makers,  not  a  valid  base  level  measure. 


5.  MISSION  CAPABLE  RATE 


Top  level  metric  for  Wing/CC.  Our  piece  of  the  pie  is 
TNMCS .  Aircraft  availability  drives  most  of  our  newer 
systems,  like  DRIVE,  so  we  need  to  watch  this  one. 

The  indicator  of  supply  effectiveness  that  receives  daily 
attention.  Greatly  simplified  with  the  creation  of  the  mass 
system.  Very  dependent  of  the  whims  of  item  managers. 

I  have  seen  collusion  between  Chiefs  of  Maintenance  and 
Supply  on  this  one,  especially  when  either  trying  for 
Daedalian  awards. 

MICAP  rates  can  be  deceiving.  Must  be  used  carefully  by  the 
COS.  Unfortunately,  others,  like  the  Wg/CC  may  use  it  to 
drive  nails  into  the  COS.  I  believe  the  MICAP  rate  can  be 
used  effectively  by  COS  for  the  internal  management  of  his 
MICAP  section. 

A  good  indicator  that  needs  to  be  further  broken  down. 

Supply  is  responsible  for  transportation  activities  and 
picks  up  this  effect  in  NMCS  Supply  time.  Need  to  hold  each 
activity  responsible  for  its  own  actions. 

The  comment  about  collusion  between  Chief  of  MX  and  COS  may 
be  valid,  but  this  whole  concern  about  collusion  at  any 
level  to  "make  the  numbers  right"  tells  us  something  about 
the  900-14  criteria  for  winning  awards,  doesn't  it. 

While  NMCS  rates  drive  a  desire  to  "place  blame,"  this,  in 
itself,  may  not  be  all  bad,  even  though  it  may  seem  so.  If 
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it  results  in  shortened  intransit  times,  or  improved 
procedures,  then  it  may  be  worth  the  pain. 


6.  TOTAL  NOT  MISSION  CAPABLE  SUPPLY 


Shared  blame  -  really  should  be  a  supply  indicator.  This  is 
a  tough  one  to  call. 

Best  indicator  of  part  of  the  NMC  rate  supply  is  responsible 
for.  To  hide  behind  maintenance  when  they  are  working  on 
the  system  doesn't  get  the  part  any  faster. 

Policy  changes  are  underway  which  will  no  longer  allow  IMs 
to  hold  parts  for  MICAPS  only.  There's  good  evidence,  and 
MAJCOMS  generally  agree,  that  mission  capability  is  better 
when  all  the  parts  are  in  the  field  and  we  let  the  bases 
redistribute  with  MASS.  IMs  have  been  conditioned  not  to  do 
this,  so  it  will  take  time  to  teach  new  behavior. 

Indicator  is  deceiving.  Total  NMCS  includes  base  level 
supply  processing  time,  transportation  time,  and  depot 
response  time.  For  the  base  level  supply  manager,  this 
provides  an  opportunity  to  get  beaten  up  for  actions  your 
not  responsible  for.  Break  this  indicator  into  its 
component  parts  and  give  a  better  perspective. 

To  me,  this  blurs  the  playing  field.  The  more  variables 
that  are  included  in  the  formula,  the  harder  it  is  to 
determine  what's  really  happening  and  hence,  how  to  control 
the  outcome . 

I  like  this  one  better  than  the  mission  capable  rate  alone. 


7.  RECOVERABLE  (CONSUMABLE)  STOCKAGE  EFFECTIVENESS 


Maybe  taking  the  stockage  effectiveness  down  to  a  level  that 
is  so  narrow  that  it  carries  little  weight. 

This  is  another  in  the  continuing  saga  of  stockage 
effectiveness.  Best  to  leave  this  indicator  in  the  capable 
hands  of  your  stock  control  supervisor. 

Will  be  trending  down  for  everyone  because  of  RSD  funding. 

We  can  only  buy,  at  most,  65%  of  condemnations.  $  available 
for  repair  are  also  dependent  on  sales.  Customers  will  be 
buying  what  helps  them  the  most,  not  necessarily  what  we 
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have  for  sale.  Supply  should  watch  this  as  a  leading 
indicator  of  the  effect  of  RSD. 

While  this  measure  should  be  a  good  indicator  as  to  how  we 
are  doing  under  RSD-  it  will  be  more  and  more  difficult  to 
manage  at  the  base  level. 

When  put  on  the  firing  line  with  a  superior,  it's  nice  to  be 
able  to  point  fingers  elsewhere.  Knowing  these  two 
different  figures  at  least  gives  me  a  pointer  where  to 
concentrate.  Useless  otherwise  for  COS. 


8.  RECOVERABLE  (CONSUMABLE)  ISSUE  EFFECTIVENESS 


Do  we  really  need  to  work  at  this  level? 

Same  comments  apply  here  as  for  overall  issue  effectiveness. 
The  customer  asks  for  what  he  needs,  not  what  we  have. 

How  far  must  this  be  carried  out? 

It's  going  to  hard  enough  to  work  effectively  with 
Consumable  Stockage  Effectiveness  under  RSD  without  having 
to  try  to  figure  out  how  meaningful  the  Issue  Effectiveness 
is  with  all  of  its  own  foibles  (similar  to  overall  issue 
effectiveness) . 


9.  RECEIPT— NOT— DUE— IN  RATE 


Good  indicator  of  possible  larger  problems  -  good  basis  for 
additional  research. 

Useful  as  a  rate  tracked  over  time.  Best  used  internally  as 
indicator  of  system  integrity.  I  have  been  in  accounts 
where  rate  was  so  low,  branch  chiefs  could  explain  each  one 
as  a  justifiable  anomaly.  Must  be  careful  not  to  make 
receipts  not  due  in  "illegal"  and  start  shooting  messengers. 
If  we  treat  people  right  they  won't  game  the  system. 

Push  levels  from  item  managers /MAJCOM  staff  requirements. 
Faulty  stock  control  management.  Customer  canceling 
requirements.  Any  number  of  reasons  could  cause  this. 
Occurrences  and  trends,  I  LOOK  AT. 

This  tells  you  if  requisitions  are  being  canceled  far  into 
the  process  of  gaining  depot  responses.  Never  a  big 
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problem.  May  best  be  expressed  as  a  ratio  of  total 
receipts ! 

This  has  never  seemed  to  be  a  big  problem,  but  it  should  be 
tracked  by  Stock  Control.  Not  sure  how  this  will  or  will 
not  be  affected  by  RSD. 


10.  PERCENT  REPAIRABLE  THIS  STATION  (OVERALL) 


This  indicator  only  tells  one  that  a  reasonable  repair 
capability  does  or  doesn't  exist  at  a  base.  Without  further 
breakdown,  which  Ch  of  Maintenance  records  show,  it  is  not  a 
good  indicator  for  probability  of  repair  at  the  location  for 
the  particular  problem  the  item  has.  It  only  indicates  to 
supply  that  they  should  investigate  further.  Not  necessarily 
a  bad  feature.  Does  it  support  the  customer  better? 

Probably  not,  if  customer  has  already  explored  the 
possibility  of  repair,  which  usually  is  a  pre-supply  process 
step. 

Under  2  Level  MX,  this  is  becoming  more  and  more 
meaningless.  However,  for  those  systems  which  are  still  3 
Level,  I  really  like  this  one.  I  compare  this  number  to 
other  bases  with  like  weapon  systems  and  work  with  MX  to 
upgrade  capabilities. 

An  indicator  that  has  seen  its  day  with  the  introduction  of 
two  level  maintenance. 

Two  level  only  applies  to  avionics  and  engines  at  this  point 
and  not  all  of  those.  There  will  still  be  a  lot  of  three 
level  maintenance  at  base  level  for  several  years. 

A  feature  of  your  maintenance  community.  Supply  is  only  one 
vehicle  through  which  this  data  can  be  derived.  Maintenance 
data  bases  provide  a  much  more  realistic  measure  of  repair 
capability  by  both  weapon  system  and  maintenance  shop 
activity.  Job  order/Work  order  specific  data  is  available 
through  CAMS. 

This  becomes  another  Source  of  Supply  that  can  make  or  break 
a  wing. 

Great  to  compare  supply  rates  to  those  maintenance  rates  for 
determining  if  the  required  updates  of  maintenance  actions 
occurred  in  the  supply  demand  data  base . 
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11.  INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 


This  is  probasly  one  of  the  best  metrics  for  continuous 
improvement  available.  I  have  seen  accounts  move  up  15  or 
20  percent  over  five  or  six  years  by  continuously  working 
the  whole  system  to  improve  accountability.  Accountability 
for  the  taxpayer's  money,  next  to  customer  support,  is  our 
most  important  task. 

We  count  it  but  we  really  don't  do  any  thing  with  it.  A 
very  frustrating  indicator  because  it  makes  inventory  look 
like  the  bad  guys. 

Important  to  internal  control  of  base  supply  assets.  The 
COS  must  remain  sensitive  to  the  fact  that  this  can  be 
manipulated  by  a  collusion  between  his  inventory  and 
warehouse  people.  Can  be  valuable  though  to  identify 
internal  problems. 

This  indicator  provides  an  opportunity  to  improve  our 
accountability.  By  examining  each  subprocess  that  leads  to 
carrying  an  item  in-stock,  we  can  further  the  trust  of 
American  taxpayers. 

I  like  this  one.  Gives  me  a  measure  of  warehouse  efficiency 
and  effectiveness  in  day-to-day  work.  Good  basic  indicator 
for  COS . 


12.  RELEVELING  FREQUENCY 


Something  to  watch,  but  not  a  metric.  Generally  close  to 
100%  of  days  available  in  my  recent  experience.  In  the  old 
days  of  1050-11,  this  was  more  important  because  of  down 
time,  etc.  Given  current  system  reliability,  you  can  make 
this  a  matter  of  policy. 

This  is  an  area  that  needs  work.  Supply  excess  is  growing 
all  the  time  due  to  auto  releveling. 

I  think  this  is  important,  if  for  nothing  else  but  to  ensure 
it  is  being  done  when  it's  supposed  to  be.  Once  you  are 
sure  this  is  being  done  frequently  enough,  it  may  be  even 
more  important  to  look  at  the  relationship  between  the  COS 
computer  room  and  the  base  communications  people.  Under  the 
new  system  where  Comm  has  the  mainframe  computer,  it  is 
EXTREMELY  important  that  supply  requisitions  are  sent  off 
base  in  a  timely  and  accurate  manner.  (We  are  finding  this 
is  not  being  done  in  all  cases,  and  that  there  is  a 
significant  learning  process  involved.) 
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Can  be  releveled  too  frequently  say  at  75%  of  stock 
authorized  instead  of  50%.  Can  cause  too  much  stock,  too 
frequent  ordering  (wasted  Transportation  $$),  etc.  Helps 
stockage  effectiveness  numbers. 

More  a  matter  of  policy  than  an  indicator  to  facilitate 
improvement . 


13.  FOLLOW-UP  FREQUENCY 


Same  comment  as  releveling  frequency.  Set  a  policy.  System 
reliability  will  do  the  rest. 

Does  this  do  any  good?  No  one  is  paying  any  attention  to  it 
because  they  don't  see  results. 

Similar  to  the  previous  category.  Make  this  a  matter  of 
policy. 

Why  overload  the  system  with  meaningless  extra  work? 
Standards  are  set,  use  them.  If  someone  is  trying  to  use 
THIS  as  a  metric,  then  they're  way  off  base.  It  only 
indicates  how  often  START  was  pushed  by  the  computer 
operator,  not  how  well  stock  control  or  any  other  unit  in 
supply  is  doing. 

This  is  important  only  if: 

-  you  are  sure  that  the  information  you  are  getting  back 
is  accurate  (not  always  the  case) 

-  you  know  what  to  do  to  correct  the  situation  WHEN  (not 
if)  you  find  that  your  requisitions  aren't  being 
handled  properly. 


14.  SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 


An  important  lower  tier  metric  supporting  inventory 
accuracy.  The  warehousing  function  needs  this  to  be  sure 
they  are  assigning  locations  to  items  stored,  which  a  first 
principle  of  accountability. 

This  one  used  to  drive  me  crazy  and  I  never  did  get  a  good 
explanation  as  to  why  it  happened. 

A  good  metric  for  your  storage  and  distribution  function. 

In  times  past,  this  was  more  or  less  a  determinant  of  when 
the  program  was  run  rather  than  an  indicator  of  vast  amounts 
of  misplaced  stock. 
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Why,  why,  why?  Not  a  major  indicator,  may  point  to 
problemsin  the  warehouse.  Definitely  becomes  a  problem  if 
there's  a  MICAP  hit  against  one  of  these  no  location  items. 

Can  be  useful  in  making  sure  the  warehouse  people  are  doing 
what  they  are  supposed  to  be  doing.  The  answers  to  the 
"why"  questions  as  they  pertain  to  this  issue  can  be 
confusing,  though.  Again,  this  is  useful  for  internal 
control  by  supply  management. 


15.  ITEM  RECORDS  PAST  DUE  INVENTORY 


A  good  sub  metric  for  the  inventory  section.  Ought  to 
approach  zero  over  time.  If  the  numbers  or  rate  are  high, 
there's  a  problem. 

Kept  inventory  on  their  toes  but  didn't  do  much  to  improve 
supply  support. 

Good  metric  for  the  Inventory  Section  supervisor.  Does 
little  to  indicate  quality  of  supply  support  at  any  higher 
level . 

This  should  happen  infrequently  if  a  good  annual  inventory 
plan  exists.  Special,  special,  special  reasons  only  for 
going  past  inventory  date.  Otherwise,  I've  got  problems. 
Could  affect  the  customer,  but  usually  doesn't.  I/Rs  frozen 
for  inventory  more  likely  to  affect  customer  temporarily, 
but  should  be  able  to  be  worked  around. 

This  used  to  be  managed  very  strictly,  but  nowadays,  as 
manpower  becomes  more  and  more  reduced,  people  are  looking 
at  this  more  openly.  It  isn't  as  important  to  inventory 
some  items  as  others,  and  this  is  being  recognized  at  all 
levels.  Policies  are  changing  to  accommodate.  At  the  same 
time,  it  will  be  more  difficult  for  the  COS  to  determine  if 
his  inventory  people  are  or  are  not  doing  their  jobs  if 
different  items  are  subject  to  different  rules.  This  is 
still  necessary  and  important. 


16.  SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  &  GSD)  STOCKAGE 
EFFECTIVENESS 


Same  comments  as  on  other  stockage  indicators 
Stock  rated  to  death. 
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Enough  is  enough! 


While  this  is  an  unpopular  indicator  (I  don't  like  it 
either),  it  may  become  more  important  under  RSD.  I  would 
not  like  to  see  it  done  away  with  until  our  systems  become 
more  well-grounded  in  the  effects  of  RSD. 


17.  SYSTEM/ GENERAL  SUPPORT  DIVISION  (SSD  &  GSD)  ISSUE 
EFFECTIVENESS 


Same  as  other  issue  effectiveness  indicators. 

Another  item  in  the  long  list  of  issue  effectiveness  rates. 
Often  used  to  explain  "Overall”  numbers. 

Same  comments  for  SSD/GSD  Stockage  Effectiveness,  except  for 
the  fact  that  I  generally  hate  issue  effectiveness  rates  for 
the  reasons  I  have  mentioned  earlier  —  namely  that  they  are 
too  easily  manipulated,  and  the  fact  that  there  are  too  many 
toads  out  there  willing  to  do  the  manipulating. 


18.  WAREHOUSE  REFUSAL  RATE 


Very  important  leading  metric  of  inventory  accuracy.  Can, 
if  managed  with  integrity,  approach  zero  over  time.  I  have 
seen  warehouses  go  for  months  without  a  refusal.  In  the 
vein  of  continuous  improvement,  when  the  numbers  get  low, 
they  can  usually  be  explained  individually.  Sometimes  it 
takes  years  to  get  to  that  point,  because  inventory  accuracy 
must  be  high  before  the  refusal  rate  approaches  zero. 

A  number  so  small  that  it  warrants  little  attention  until  it 
goes  totally  out  of  specs. 

This  approaches  zero.  To  pursue  continuous  improvement, 
this  measurement  should  change  to  reflect  individual 
incidents.  Improvements  found  by  slicing  through  the 
processes  that  caused  the  refusal  and  tracing  down  the 
origin  of  the  problem. 

Agree  that  it  is  a  part  of  inventory  accuracy.  But  it  isn't 
the  whole  ball  game.  Taken  with  serviceable  balance  w/no 
location  and  RVPs  and  their  reasons,  over  time,  COS  has  a 
good  feel  for  what's  truly  happening.  This  is  an  indicator 
of  NON-customer  support.  We  didn't  give  him  the  item. 


although  we  promised  and  lost  valuable  time  either  in 
inventory  or  backordering. 

I  liked  this  indicator  as  a  COS,  not  because  of  the  low 
numbers  ordinarily  associated  with  it,  but  it  gave  me  reason 
to  "peel  the  onion"  so  to  speak,  and  delve  deeper  into  why 
things  were  or  were  not  happening  in  the  warehouse.  Again, 
this  is  one  which  MUST  be  handled  carefully  and  with 
integrity  because  of  the  ease  with  which  a  slick 
superintendent  can  slide  over  and  around  the  real  reasons 
for  problems  in  this  area. 


19.  GENERAL  COMMENTS 


I  believe  the  Reverse  Post  rates  are  one  of  the  most 
important  indicator  available  to  the  COS.  Training 
requirements,  DIREP  identification,  etc.,  etc.,  etc.  SUPER 
IMPORTANT ! 

Let's  add  reverse  post  rates!  Tells  COS  of  training 
problems,  system  difficulties,  and  coordination  problems. 

Suggest  Reverse  Post  rate  be  included  as  an  internal  supply 
metric  to  drive  training  needs,  system  anomalies,  etc. 

Develop  a  survey  that  would  indicate  satisfaction  levels  of 
customer  service.  Each  supply  customer  organization  would 
periodically  get  the  opportunity  to  express  thoughts  on  the 
responsiveness  of  supply  support,  personal  attention,  and 
problem  solving  efficiency.  Use  statistical  analysis  to 
determine  areas  where  increased  awareness  might  be 
necessary. 

Base  Supply  must  face  the  harsh  reality  that  it  can  no 
longer  support  every  customer  request  off  the  shelf.  It 
needs  to  look  at  ways  to  improve  customer  support  even  if  it 
means  taking  supply  out  of  the  process. 

Management  products  are  geared  to  evaluate  the  base  supply 
system  by  the  numbers,  it  does  not  look  at  innovative  ways 
to  make  the  system  better. 

To  go  along  with  the  management  products,  there  should  be  an 
answer  sheet  that  gives  workers  possible  solution  to  improve 
the  process. 

The  financial  world  will  dominate  further  supply  operations, 
but  it  is  the  most  confusing  aspect  of  the  entire  base 
supply  system. 
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Excess  management  should  be  a  management  product 


While  not  perfect  by  any  means,  M32  data  provides  COS  and 
staff  valuable  insights  into  the  account.  As  stated  by  one 
participant,  the  data  can  be  used  by  all  concerned  for 
learning  purposes.  Few  of  these  indicators  were  derived  to 
indicate  positive/negative  customer  support  trends.  In  the 
world  of  TQM,  few  of  the  supply  indicators  drive  actual 
process  improvements  -  they  are  after— the-process  inspection 
points,  not  mid-cycle  check  points  for  process  health.  In 
fact,  by  the  time,  negative  trends  develop,  you  might  as 
well  carry  it  all  in  a  hand  basket. 

I,  and  a  lot  of  others  I  have  talked  with  lately,  have  some 
real  problems  with  the  way  the  Air  Force  level  awards  are 
determined.  The  emphasis  has  always  been  on  either  numbers 
or  a  combination  of  numbers  and  self  improvement  efforts. 

The  past  two  years,  "numbers”  has  been  the  bugaboo.  It  is 
much  easier  for  a  base  with  a  high  density  fleet  to  compete 
based  on  numbers.  This  effectively  eliminates  all  bases 
which,  although  they  may  have  an  extremely  good  operation, 
just  can't  make  it  due  to  a  low  density  fleet,  experimental 
aircraft  which  notoriously  have  a  large  amount  of  down  time, 
etc.  I  believe  that  any  future  efforts  to  come  up  with 
"meaningful  measures  of  merit"  should  be  geared  to  JUST 
THAT.  How  do  you  measure  a  wing  to  see  if  they  are  REALLY 
doing  a  good  job,  or  if  they  are  merely  blowing  smoke?  Or, 
how  do  you  effectively  compare  low  and  high  density  orgs? 

I  think  that  there  may  need  to  be  two  types  of  measurement 
tools  available  to  the  COS  —  one  which  he  can  use 
internally  that  will  mean  something  to  "the  non-manipulative 
manager, "  and  another  which  can  be  used  to  numerically  rate 
mission-diverse  organizations. 

I/Rs  frozen  for  inventory  might  be  a  viable  measure  of  non¬ 
customer  support,  provided  it  was  compiled  more  frequently 
than  monthly,  and  most  management  levels  of  Supply  looked  at 
routinely. 


NOTES  FROM  OPEN  DISCUSSION 

#1  -  computer  determines  levels 

-  doesn't  tell  how  well  we  support  our  customer 

-  doesn't  show  movement  of  supplies 

-  good  data  for  very  senior  MAJCOM  officials 

-  base  level  no  control 

-  funding  driven 

-  dead  item  records  kept  for  manning,  drops  s/e 
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#2  -  combine  i/e  and  s/e  on  same  chart  =  much  better 

-  need  a  proactive  measure 

-  potential  good  internal  measure 

-  manipulation  »  integrity  problem 

-  loosing  customers  because  of  "competition" 

#3  -  bench  stock  is  confusing  for  customers 

-  obligation  authority 

-  currently  buying  65%  rsd  (58%  ssd)  condemnations 

-  used  for  "horn  tooting"  for  supply 

-  service  rating  might  be  better  than  availability 

-  money  is  first  restricted  on  bench  stock  when  funds 
become  tight  and  first  to  be  spent  when  money  becomes 
available 

#4  -  much  better  than  #1 

-  take  care  of  critical  customers 

-  take  away/change  the  85%  goal  imposed  by  MAJCOMs 

-  effected  by  funding  and  reduced  #  of  airframes 

-  what  level  of  activity  is  needed  to  support  wartime 
capability 

-  distinguish  between  weapon  systems 

-  rsd  'sale'  is  pushed  as  opposed  to  'stock' 

#5  -  controlled  by  item  managers  who  want  to  hoard  parts, 

thereby  creating  MICAPs 

#6  -  truer  than  #5 

-  break  it  down  into  component  parts  to  improve  the 
process 

-  what  about  transportation  time? 

-  mode  of  transportation  is  not  really  important,  just 
when  it  will  arrive 

-  clean  it  up  a  little 

-  difficult  to  track  trans  between  many  different 
commercial  carriers 

#7  -  many  of  these  type  indicators  are  very  good  for 

internal  training 

#8  -  ditto 

#9  -  ratio  -  ratio  -  ratio 

-  #s  so  small  that  you  have  to  look  for  spikes, 
anomalies 

-  useful 

-  shows  where  training  is  needed 

-  defense  programs  sometimes  are  factors 

#10  -  repairs  at  base  level  will  still  be  around  for  awhile 

-  this  is  tracked  by  supply  because  of  ability  to  track 

-  use  as  a  probe  to  compare  with  other  bases 

-  no  control 
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-  less  specialization  makes  overlap  important  in 
logistics 

-  systems  perspective 

-  demand  history  is  told  by  this  indicators 

#11  -  can  be  made  to  appear  misleading 

-  any  #  generated  by  a  system  can  be  manipulated  (even 
this) 

#12  -linked  to  s/e  (overall) 

-  automatically  done  by  the  computer,  too  often 
overworks  and  overstocks  stuff 

-  SIFS 

-  communication  with  COMM/teamwork 
#13  -  similar  to  #12 

-  must  be  used  intelligently  to  determine  if  there' s  a 
problem  at  a  depot 

-  good  to  be  used  in  analysis  of  results,  not  just 
follow-up  frequency 

-  use  by  depot/im/system 

#14  -  system  and  quirk  controlled 

-  good  internal  measure 

#15  -  good  internal  measure 

-  still  necessary  and  important  internally 

-  i/r  frozen  for  inventory  relates  to  customer  support 

#18  -  well  liked  by  participants 

-  good  indicator  of  poor  customer  support 

-  can  be  manipulated  by  a  "slick"  superintendent 

#19  -  reverse  posts!!! 

-  $$$ 

-  ex  post  facto 

-  need  measures  comparable  between  different  types  of 
bases,  different  commands,  and  different  fleets 

-  limit  the  indicator  that  are  difficult  to  manipulate 

-  don't  let  metric  be  used  as  a  hammer 
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Appendix  C:  Ratings  From  First  Focus  Group 


»  Equally  Weighted  Criteria  (Text  Report)  « 


Median 

Mean 

17 

16.4 

16 

16.0 

15 

14.0 

14 

14.6 

14 

12.8 

13 

11.0 

12 

13.0 

12 

12.4 

11 

9.4 

10 

9.6 

10 

9.2 

8 

9.0 

7 

7.8 

6 

7.0 

6 

6.8 

5 

7.6 

4 

5.8 

4 

4.4 

**  ALTERNATIVES  ** 


WAREHOUSE  REFUSAL  RATE 

TOTAL  NOT  MISSION  CAPABLE  SUPPLY 

BENCH  STOCK  AVAILABILITY 

STOCKAGE  EFFECTIVENESS  (PRIMARY  WEAPON  SYSTEM) 
INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 
STOCKAGE  EFFECTIVENESS  -  (OVERALL) 

MISSION  CAPABLE  RATE 

ISSUE  EFFECTIVENESS  -  (OVERALL) 

SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 
ITEM  RECORDS  PAST  DUE  INVENTORY 
PERCENT  REPAIRABLE  THIS  STATION  (OVERALL) 
RECEIPT-NOT-DUE-IN  RATE 

RECOVERABLE  (CONSUMABLE)  STOCKAGE  EFFECTIVENESS 
SSD/GSD  STOCKAGE  EFFECTIVENESS 
SSD/GSD  ISSUE  EFFECTIVENESS 

RECOVERABLE  (CONSUMABLE)  ISSUE  EFFECTIVENESS 
RELEVELING  FREQUENCY 
FOLLOW-UP  FREQUENCY 


»  Criteria  Matrix  « 

1.  How  well  does  this  indicator  reflect  customer/mission  support? 

Participant  Ratings 


**  Alternative  ** 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

N 

n 

MN 

STD 

MED 

STOCKAGE  EFFECT IVENE 

— 

— 

— 

— 

2 

— 

2 

1 

— 

— 

5 

5 

6.4 

1.34 

7 

ISSUE  EFFECTIVENESS 

- 

- 

- 

- 

1 

1 

1 

- 

2 

- 

5 

5 

7.2 

1.79 

7 

BENCH  STOCK  AVAILABI 

- 

- 

1 

- 

- 

- 

1 

3 

- 

- 

5 

5 

6.8 

2.17 

8 

STOCKAGE  EFFECTIVENE 

- 

- 

- 

- 

1 

- 

- 

2 

2 

- 

5 

5 

7.8 

1.64 

8 

MISSION  CAPABLE  RATE 

- 

- 

- 

- 

1 

- 

2 

1 

1 

- 

5 

5 

7.2 

1.48 

7 

TOTAL  NOT  MISSION  CA 

- 

- 

- 

- 

- 

- 

1 

3 

1 

- 

5 

5 

8.0 

0.71 

8 

RECOV/CONS  STK  EFFEC 

- 

1 

2 

1 

- 

- 

- 

1 

- 

- 

5 

5 

4.0 

2.35 

3 

RECOV/CONS  ISSUE  EFF 

- 

1 

2 

1 

- 

- 

- 

1 

- 

- 

5 

5 

4.0 

2.35 

3 

RECEIPT-NOT-DUE-IN 

- 

1 

2 

2 

- 

- 

- 

- 

- 

- 

5 

5 

3.2 

0.84 

3 

PERCENT  REPAIRABLE 

- 

1 

1 

- 

2 

- 

- 

1 

- 

- 

5 

5 

4.6 

2.30 

5 

INVENTORY  ACCURACY 

- 

1 

- 

1 

- 

- 

2 

- 

- 

1 

5 

5 

6.0 

3.08 

7 

RELEVELING  FREQUENCY 

1 

2 

1 

1 

- 

- 

- 

- 

- 

- 

5 

5 

2.4 

1.14 

2 

FOLLOW-UP  FREQUENCY 

1 

3 

- 

1 

- 

- 

- 

- 

- 

5 

5 

2.2 

1.10 

2 

SERVICEABLE  BALANCE 

- 

- 

- 

2 

1 

- 

- 

- 

- 

5 

5 

4.0 

1.87 

5 

ITEM  RECORDS  PAST  DUE 

- 

r> 

1 

- 

1 

- 

1 

- 

- 

- 

5 

5 

3.8 

2.17 

3 

SSD/GSD  STOCK  EFF 

- 

i 

1 

2 

- 

1 

- 

- 

- 

- 

5 

5 

3.8 

1.48 

4 

SSD/GSD  ISSUE  EFF 

- 

i 

2 

- 

1 

1 

- 

- 

- 

- 

5 

5 

3.8 

1.64 

3 

WAREHOUSE  REFUSAL 

- 

- 

- 

- 

1 

- 

- 

1 

2 

1 

5 

5 

8.2 

1.92 

9 

116 


2.  How  well  does  this  indicator  lead  you  to  continually  improve  the 
process 


Participant  Ratings 


**  Alternative  ** 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

N 

n 

MN 

STD 

MED 

STOCKAGE  EFFECTIVENE 

1 

1 

— 

— 

1 

2 

_ 

_ 

— 

5 

5 

4.6 

2.88 

6 

ISSUE  EFFECTIVENESS 

- 

1 

1 

- 

1 

- 

1 

- 

1 

- 

5 

5 

5.2 

2.86 

5 

BENCH  STOCK  AVAILABIL 

- 

- 

- 

- 

- 

1 

2 

2 

- 

- 

5 

5 

7.2 

0.84 

7 

STOCKAGE  EFFECTIVENE 

- 

- 

- 

- 

1 

2 

- 

1 

1 

- 

5 

5 

6.8 

1 . 64 

6 

MISSION  CAPABLE  RATE 

- 

- 

1 

- 

2 

- 

1 

- 

1 

- 

5 

5 

5.8 

2.28 

5 

TOTAL  NOT  MISSION  CAP 

- 

- 

- 

- 

- 

- 

1 

3 

1 

- 

5 

5 

8.0 

0.71 

8 

RECOV/CONS  STOCK  EFF 

- 

2 

- 

2 

- 

- 

1 

- 

- 

- 

5 

5 

3.8 

2.05 

4 

RECOV/CONS  ISSUE  EFF 

- 

2 

1 

1 

- 

- 

1 

- 

- 

- 

5 

5 

3.6 

1.85 

3 

RECEIPT— NOT— DUE -IN 

- 

1 

- 

- 

2 

- 

1 

- 

- 

1 

5 

5 

5.8 

2.95 

5 

PERCENT  REPAIRABLE 

1 

1 

- 

- 

1 

1 

- 

- 

1 

- 

5 

5 

4.6 

3.21 

5 

INVENTORY  ACCURACY 

- 

- 

1 

- 

- 

1 

- 

2 

1 

- 

5 

5 

6.8 

2.39 

8 

RELEVELING  FREQUENCY 

2 

1 

- 

1 

- 

- 

- 

- 

1 

- 

5 

5 

3.4 

3.36 

2 

FOLLOW-UP  FREQUENCY 

2 

1 

1 

1 

- 

- 

- 

- 

- 

- 

5 

5 

2.2 

1.30 

2 

SERVICEABLE  BALANCE 

- 

- 

1 

1 

- 

1 

2 

- 

- 

- 

5 

5 

5.4 

1.82 

6 

ITEM  RECORDS  PAST  DUE 

- 

- 

1 

- 

1 

- 

3 

- 

- 

- 

5 

5 

5.8 

1.79 

7 

SSD/GSD  STOCK  EFF 

- 

3 

- 

1 

- 

1 

- 

- 

- 

- 

5 

5 

3.2 

1.79 

2 

SSD/GSD  ISSUE  EFF 

- 

3 

1 

- 

- 

1 

- 

- 

- 

- 

5 

5 

3.0 

1.73 

3 

WAREHOUSE  REFUSAL 

- 

- 

- 

- 

- 

- 

1 

3 

- 

1 

5 

5 

8.2 

1.10 

8 

»  Alternative  Matrix  « 

1.  STOCKAGE  EFFECTIVENESS  -  (OVERALL) 


Participant 

Ratings 

1 

■  .r-mxl-m. 

**  Criteria  ** 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

N 

n 

MN 

STD 

MD 

Customer/Mi33ion  Focu3 

_ 

_ 

_ 

2 

2 

1 

_ 

_ 

5 

5 

6.40 

1.34 

7.00 

Continuous  Process  Impro 

1 

1 

' 

1 

2 

" 

5 

5 

4.60 

2.88 

6.00 

2.  ISSUE  EFFECTIVENESS  - 

(OVERALL) 

Participant 

Ratings 

**  Criteria  ** 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

N 

n 

MN 

STD 

MD 

Customer/Mission  Focu3 

_ 

_ 

_ 

1 

1 

1 

_ 

2 

_ 

5 

5 

7.20 

1.79 

7.00 

Continuous  Process  Impro 

1 

1 

1 

1 

" 

1 

" 

5 

5 

5.20 

2.86 

5.00 

3.  BENCH  STOCK  AVAILABILITY 

Participant 

Ratings 

**  Criteria  ** 

1 

2 

3 

4 

5 

S 

7 

8 

9 

10 

N 

n 

MN 

STD 

MD 

Customer/Mission  Focus 

- 

- 

1 

- 

- 

- 

1 

3 

_ 

_ 

5 

5 

6.80 

2.17 

8.00 

Continuous  Process  Impro 

1 

2 

2 

" 

" 

5 

5 

7.20 

0.84 

7.00 

4.  STOCKAGE  EFFECTIVENESS 

(PRIMARY 

WEAPON  SYSTEM) 

Participant 

Ratings 

**  Criteria  ** 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

N 

n 

MN 

STD 

MD 

Customer/Mission  Focus 

- 

- 

- 

- 

1 

- 

- 

2 

2 

_ 

5 

5 

7.80 

1.64 

8.00 

continuous  Process  Impro 

- 

1 

2 

- 

1 

1 

- 

5 

5 

6.80 

1.64 

6.00 

117 


5.  MISSION  CAPABLE  RATE 


Participant  Ratings 


**  Criteria  ** 

1 

2 

3 

4  5  6 

7  8  9  10 

N 

n 

MN 

STD 

MD 

Customer/Mission  Focus 

1 

2 

1 

1  -  - 

_  -  —  _ 

5 

5 

2.40 

1.14 

2.00 

Continuous  Process  Impro 

2 

1 

- 

1  -  - 

-  -  1  - 

5 

5 

3.40 

3.36 

2.00 
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13.  FOLLOW-UP  FREQUENCY 


Participant 

Ratings 

**  Criteria  ** 

1 

2 

3  4  5 

6  7  8  9 

10 

N 

n 

MN 

STD 

MD 

Cuatomer/Mlsslon  Focus 

1 

3 

-  1  - 

-  —  —  - 

- 

5 

5 

2.20 

1.10 

2.00 

Continuous  Process  Impro 

2  - 

1 

1  1  - 

5 

5 

2.20 

1.30 

2.00 

14.  SERVICEABLE  BALANCE  WITH 

NO 

WAREHOUSE  LOCATION 

Participant 

Ratings 

**  Criteria  ** 

1 

2 

3  4  5 

6  7  8  9 

10 

N 

n 

'IN 

STD 

MD 

Customer/Mission  Focu3 

2 

-  -  2 

1  -  -  - 

_ 

5 

5 

4.00 

1.87 

5.00 

Continuous  Process  Impro 

1  1  - 

1  2  -  - 

- 

5 

5 

5.40 

1.82 

6.00 

15.  ITEM  RECORDS  PAST  DUE 

INVENTORY 

Participant 

Ratings 

**  Criteria  ** 

1 

2 

3  4 

5 

6 

7  8 

9 

10 

N 

n 

MN 

STD 

MD 

Customer/Mission  Focus 

_ 

2 

1  - 

1 

— 

1  - 

- 

- 

5 

5 

3.80 

2.17 

3.00 

Continuous  Process  Impro 

' 

1  - 

1 

" 

3  - 

' 

5 

5 

5.80 

1.79 

7.00 

16.  SYSTEM/ GENERAL  SUPPORT  DIVISION 

(SSD 

& 

GSD)  STOCKAGE 

EFFECTIVENESS 

Participant 

Ratings 

*»  Criteria  ** 

1 

2 

3  4 

5 

6 

7  8 

9 

10 

N 

n 

MN 

STD 

MD 

Customer/Mission  Focus 

— 

1 

1  2 

_ 

1 

-  - 

_ 

_ 

5 

5 

3.80 

1.48 

4.00 

Continuous  Process  Impro 

3 

-  1 

' 

1 

" 

5 

5 

3.20 

1.79 

2.00 

17 .  SYSTEM/GENERAL  SUPPORT  DIVISION 

(SSD 

& 

GSD)  : 

ISSUE  EFFECTIVENESS 

Participant 

Ratings 

**  Criteria  ** 

1 

2 

3  4 

5 

6 

7  8 

9 

10 

N 

n 

MN 

STD 

MD 

Customer/Mission  Focus 

_ 

1 

2  - 

1 

1 

_  _ 

- 

- 

5 

5 

3.80 

1.64 

3.00 

Continuous  Process  Impro 

- 

3 

1  - 

- 

1 

-  * 

- 

- 

5 

5 

3.00 

1.73 

2.00 

18.  WAREHOUSE  REFUSAL  RATE 


Participant  Ratings 


**  Criteria  **  12 

3  4  5 

6  7 

8 

9 

10 

N 

n 

MN 

STD 

MD 

Customer/Mi3slon  Focus  -  - 

-  -  1 

_  _ 

1 

2 

1 

5 

5 

8.20 

1.92 

9.00 

Continuous  Process  Impro  -  - 

“  ~  - 

-  1 

3 

“ 

1 

5 

5 

8.20 

1.10 

8.00 

»  Average  ratings  of  all  alternatives  on  each  criterion  « 


MN 

STD 

Criterion 

5.20 

5.10 

1.99 

1.83 

How  well  does 

How  well  does 

this 

thi3 

indicator 

indicator 

reflect  customer/mission  support? 

lead  you  to  continually  improve? 
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HOW  WELL  DOES  THIS  INDICATOR  REFLECT  COSTOMER/MISSION  SUPPORT? 


Ratings 

(MEAN) 


Alternatives 

WAREHOUSE  REFUSAL  RATE 

TOTAL  WOT  MISSION  CAPABLE  SUPPLY 

STOCKAGE  EFFECTIVENESS  (PRIMARY  WEAPON  SYSTEM) 

MISSION  CAPABLE  RATE 

ISSUE  EFFECTIVENESS  -  (OVERALL) 

BENCH  STOCK  AVAILABILITY 
STOCKAGE  EFFECTIVENESS  -  (OVERALL) 

INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 

PERCENT  REPAIRABLE  THIS  STATION  (OVERALL) 

SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 
RECOVERABLE  (CONSUMABLE)  STOCKAGE  EFFECTIVENESS 
RECOVERABLE  (CONSUMABLE)  ISSUE  EFFECTIVENESS 
ITEM  RECORDS  PAST  DUE  INVENTORY 

SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  &  GSD)  STOCKAGE  EFFECTIVEN 
SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  i  GSD)  ISSUE  EFFECTIVENESS 
RECEIPT-NOT-DUE-IN  RATE 
RELEVELING  FREQUENCY 
FOLLOW-UP  FREQUENCY 


HOW  WELL  DOES  THIS  INDICATOR  LEAD  YOU  TO  CONTINUALLY  IMPROVE  THE  PROCESS? 


Ratings 

(MEAN) 


(MEAN)  Alternatives 

8.20  WAREHOUSE  REFUSAL  RATE 

8.00  TOTAL  NOT  MISSION  CAPABLE  SUPPLY 

7.20  BENCH  STOCK  AVAILABILITY 

6.80  STOCKAGE  EFFECTIVENESS  (PRIMARY  WEAPON  SYSTEM) 

6.80  INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 

5.80  MISSION  CAPABLE  RATE 

5.80  ITEM  RECORDS  PAST  DUE  INVENTORY 

5 . 80  RECEIPT-NOT-DUE-IN  RATE 

5.40  SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 

5.20  ISSUE  EFFECTIVENESS  -  (OVERALL) 

4.60  STOCKAGE  EFFECTIVENESS  -  (OVERALL) 

4.60  PERCENT  REPAIRABLE  THIS  STATION  (OVERALL) 

3.80  RECOVERABLE  (CONSUMABLE)  STOCKAGE  EFFECTIVENESS 

3.60  RECOVERABLE  (CONSUMABLE)  ISSUE  EFFECTIVENESS 

3.40  RELEVELING  FREQUENCY 

3.20  SYSTEM/ GENERAL  SUPPORT  DIVISION  (SSD  S  GSD)  STOCKAGE  EFFECTIVEN 

3.00  SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  &  GSD)  ISSUE  EFFECTIVENESS 

2.20  FOLLOW-UP  FREQUENCY 

HOW  WELL  DOES  THIS  INDICATOR  REFLECT  CUSTOMER/MISSION  SUPPORT? 


Ratings 

(MEDIAN) 


Alternatives 


WAREHOUSE  REFUSAL  RATE 

TOTAL  NOT  MISSION  CAPABLE  SUPPLY 

STOCKAGE  EFFECTIVENESS  (PRIMARY  WEAPON  SYSTEM) 

BENCH  STOCK  AVAILABILITY 

MISSION  CAPABLE  RATE 

ISSUE  EFFECTIVENESS  -  (OVERALL) 

STOCKAGE  EFFECTIVENESS  -  (OVERALL) 

INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 

PERCENT  REPAIRABLE  THIS  STATION  (OVERALL) 

SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 

SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  &  GSD)  STOCKAGE  EFFECTIVEN 
RECOVERABLE  (CONSUMABLE)  STOCKAGE  EFFECTIVENESS 
RECOVERABLE  (CONSUMABLE)  ISSUE  EFFECTIVENESS 

SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  S  GSD)  ISSUE  EFFECTIVENESS 

ITEM  RECORDS  PAST  DUE  INVENTORY 

RECEIPT-NOT-DUE-IN  RATE 

RELEVELING  FREQUENCY 

FOLLOW-UP  FREQUENCY 
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HOW  WELL  DOES  THIS  INDICATOR  LEAD  YOU  TO  CONTINUALLY  IMPROVE  THE  PROCESS? 

Ratings 

(MEDIAN)  Alternatives 

8.00  WAREHOUSE  REFUSAL  RATE 

8.00  TOTAL  NOT  MISSION  CAPABLE  SUPPLY 

7.00  INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 

7.00  BENCH  STOCK  AVAILABILITY 

7.00  ITEM  RECORDS  PAST  DUE  INVENTORY 

6.00  STOCKAGE  EFFECTIVENESS  (PRIMARY  WEAPON  SYSTEM) 

6.00  SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 

6.00  STOCKAGE  EFFECTIVENESS  -  (OVERALL) 

5.00  MISSION  CAPABLE  RATE 

5.00  RECEIPT -NOT— DUE- IN  RATE 

5.00  ISSUE  EFFECTIVENESS  -  (OVERALL) 

5.00  PERCENT  REPAIRABLE  THIS  STATION  (OVERALL) 

4.00  RECOVERABLE  (CONSUMABLE)  STOCKAGE  EFFECTIVENESS 

3.00  RECOVERABLE  (CONSUMABLE)  ISSUE  EFFECTIVENESS 

3.00  SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  &  GSD)  ISSUE  EFFECT  I '/ENF 

2.00  RELEVELING  FREQUENCY 

2.00  SYSTEM/GENERAL  SUPPORT  DIVISION  (SSD  &  GSD)  STOCKAGE  EFFECTIVEN 

2 . 00  FOLLOW-UP  FREQUENCY 
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Appendix  D:  Revised  Indicators 


GENERAL  INFORMATION 


The  revised  supply  management  indicators  presented  to 
the  second  focus  group  are  listed  below  along  with  the 
formulas  used  to  equate  the  indicators. 


#1  -  UNREPORTED  EXCESS  INVENTORY  RATIO 

$  Value  of  Unreported  Excess  Materiel 
Total  $  Value  of  Inventory 


#2  -  STOCKAGE  EFFECTIVENESS  -  (EACH  WEAPON  SYSTEM) 

_ No.  WS  Units  Issued _ 

No .  ws  Uni ts  Issued  +  No.  WS  Uni ts  B/ O  -  No.  ws  Uni ts  B/ O  4.W 


#3  -  INVENTORY  EFFECTIVENESS  RATIO  -  (OVERALL) 


Stockage  Effectiveness  x 


$  Value  of  On-hand  Inventory 
$  Value  of  Authorized  Inventory 


#4  -  ISSUE  EFFECTIVENESS  -  (EACH  WEAPON  SYSTEM) 

_ No.  WS  Units  Issued _ 

No.  WS  Units  Issued  +  No.  WS  Units  B/O 


#5  -  NOT  MISSION  CAPABLE  SUPPLY  RATE 

_ NMCS  Hours _ 

Total  Possessed  Hours 

#6  -  TOTAL  NOT  MISSION  CAPABLE  SUPPLY 

NMCS  +  NMCB  Hours 
Total  Possessed  Hours 
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#7  -  RECEIPT-NOT— DUE-IN  RATE 


Total  Shipments  Received  With  No  Corresponding  Due-in 
Total  Number  of  Receipts 


#8  -  PERCENT  REPAIRABLE  THIS  STATION  (3-LEVEL  MX  ONLY) 

Number  3  Level  Maintenance  Units  Repairable  This  Station 
No.  3 LM  Units  RTS  -  No.  3 LM  Units  NRTS  -  No.  3 LM  Units  Cond 


#9  -  INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 

Total  Units  Over  +  Short 
Total  Record  Balance 


#10  -  SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 

Numer  of  Item  Records  With  Serv  Bal  and  No  Whse  Location 
Total  Number  of  Item  Records 


#11  -  ITEM  RECORDS  FROZEN  FOR  INVENTORY  RATE 

Item  Records  Frozen  for  Non-routine  Inventory 
Total  Item  Records 


#12  -  TIME  TO  CLEAR  FROZEN  ITEM  RECORDS 

Average  Time  to  Clear  Non-routine  Frozen  Item  Records 


#13  -  REVERSE  POST  RATE 

Number  of  Reverse  Post  Transactions 
Total  Number  of  Transactions 


#14  -  WAREHOUSE  REFUSAL  RATE 

Total  Number  of  Warehouse  Refusals 
Total  Number  of  Issues 
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#15  -  BENCH  STOCK  AVAILABILITY  (BY  ORGANIZATION) 

Total  Line  Items  Authorized  -  Line  Items  Due -out 
Total  Lina  Items  Authorized 


#16  -  NOT-STOCKED  BACKORDER  RATIO 

Number  of  Units  Backordered  4 W 
Total  Number  of  Units  Backordered 


#17  -  BENCH  STOCK-MI CAP  RATE 

Number  of  MICAP  Hours  Due  to  Bench  Stocked  Items 
Total  Number  of  MICAP  Hours 


#18  -  SUPPLY  CANN  RATE 

Number  of  CANN  Actions  to  Clear  +  Prevent  MICAPs 
Total  Number  of  CANN  Actions 
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Appendix  E:  Comments  From  Second  Focus  Group 


UNREPORTED  EXCESS  INVENTORY  RATIO 


This  would  give  a  supply  manager  some  feel  for  how 
reactive  the  computer (for  automatic  excess  reporting)  or  the 
stock  control  section (for  manual  excess  id  and  reporting) 
may  be.  Provides  an  opportunity  to  tweak  either  one. 

This  merely  gives  you  an  idea  of  what's  waiting  in  the  wings 
for  disposal.  If  stratified  in  several  different  ways,  it 
could  offer  a  basis  to  discuss  ordering  habits  with 
customers,  but  here  would  be  touchy  territory  as  supply 
finally  got  out  of  the  "Policeman's"  role  it  once  had. 
Innovative  supply  guys  could  offer  discount  bargains  to  try 
and  recoup  some  of  the  dollars  invested.  Any  monies  gained 
is  better  than  none  or  incurring  transportation  costs  to 
ship  elsewhere. 

Like  the  idea  of  a  ratio.  Measures  results  rather  than 
activity . 

I  like  the  idea  of  looking  at  "unreported"  excess,  but  we 
still  need  to  be  viewing  excess  which,  although  it  may  have 
been  reported,  is  still  on  hand.  We  need  to  get  a  firm 
handle  on  excess  which  we  have  identified  and  reported,  but 
which  is  still  hanging  around  for  whatever  reason.  (I 
recently  read  a  report  which  "inadvertently"  identified 
excess  which  had  been  reported  but  which  had  continued  to 
sit  in  base  supply  for  12—18  months  —  with  no  end  in 
sight.)  The  report  didn't  seem  to  indicate  a  problem  with 
this  because  "it  had,  in  fact,  been  reported."  However,  I 
see  this  as  a  real  problem  because  it's  tying  up 
needed  dollars  needed  to  run  the  account. 

Any  action  to  identify  excess  is  a  positive  initiative. 
Millions  of  dollars  of  usable  supply  assets  are  hidden  away 
in  supply  warehouses  just  waiting  to  be  identified  through 
the  current  system.  Excesses  continue  to  grow  especially 
with  the  rapid  changes  to  computers  and  software.  By  the 
time  Supply  builds  up  demand  data,  the  current  version  is 
obsolete.  Continue  to  make  getting  rid  of  excess  easier. 


2.  STOCKAGE  EFFECTIVENESS  (EACH  WEAPON  SYSTEM) 


This  indicator  is  not  necessarily  a  good  tool  for  continuous 
improvement.  However,  it  is  vital  information  necessary  to 
protect  one's  hind  quarter. 
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At  least  this  gives  some  visibility  by  WS.  Total  figure  for 
the  account  is  meaningless  when  trying  to  enhance  a  WS' s 
support.  Given  the  AF's  stockage  formulas,  there  isn't  much 
mechanically  that  can  be  done  to  enhance  the  support.  It 
will  still  be  ar.  intervention  effort  by  stock  controllers. 

This  measure  tells  how  well  the  depots  are  supporting.  At 
least  one  MAJCOM  is  stratifying  this  by  weapon  system  and 
depot.  Tells  them  where  they  need  to  concentrate  their 
follow-up  work. 

Agree  with  above  statements.  "By  weapon  system"  should  be  a 
much  better  indicator  of  not  only  the  ability  of  the  COS  to 
support  the  WS,  but  also  show  where  management  emphasis  is 
needed. 

Good  approach  because  it  brings  stockage  effectiveness  down 
to  a  more  visible  management  tool.  Helps  identify  which 
depot  are  really  working  to  support  their  weapon  system. 


3.  INVENTORY  EFFECTIVENESS  RATIO  -  (OVERALL) 


I  like  this  indicator  expressed  as  a  ratio.  Gives  the 
supply  manager  an  opportunity  to  steer  rather  than  row  this 
ship . 

I'd  like  to  see  a  couple  of  different  measures  stratified 
with  this. 

1.  stkage/issue  effectiveness  *  ($  value  of  non-excess 
inventory/$  value  of  authorized  inventory) 

2.  stkage/issue  effectiveness  *  ($  value  of  reported  and 
unreported  excess  inventory/$  value  of  authorized 
inventory) . 

REASON:  How  well  is  my  non-excess  working  for  me?  How  much 

does  my  excess  hamper  my  effectiveness  at  providing  what  my 
customer  needs? 

Agree  with  comments  on  stratification.  Need  to  define  what 
behavior  this  would  drive.  If  it  doesn't  drive  a  desired 
behavior,  then  it  isn't  an  effective  metric.  I  think  this 
would  drive  the  supply  manager  to  increase  stockage 
effectiveness,  or  increase  on  hand  inventory,  or  decrease 
authorized  inventory.  Any  one  of  those  might  be  undesirable 
or  desirable,  depending  on  the  circumstances. 

Agree  with  stratification  and  whether  or  not  this  measure 
would  be  useful  to  the  COS.  I'm  concerned  about  using  ONLY 
dollar  values.  Some  very  high  dollar  value  parts  might  skew 
the  ratio  and  make  the  measure  meaningless  —  or  at  least 
less  useful.  I'm  concerned  that  this  might  cause  management 
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to  take  unnecessary  action  or  the  "wrong"  action  to  correct 
something  which  may  not  be  broken. 

Based  on  the  large  dollar  values  involved  with  this  measure, 
can  a  Supply  squadron  really  impact  this  ratio.  Some  good 
gee  whiz  info  but  of  limited  value  and  may  only  generate 
over  management  because  we, re  not  meeting  the  standard. 


4.  ISSUE  EFFECTIVENESS  -  (EACH  WEAPON  SYSTEM) 


So  much  of  the  decision  regarding  when  to  stock  or  how  much 
to  stock  has  become  automated.  Can  this  indicator  provide 
an  avenue  for  improvement  or  are  we  once  again  trying  to 
protect  ourselves.  Nice  to  know,  that's  about  all.  If  we 
could  change  the  customer  service  level  in  the  demand 
formula  computations  (to  something  other  than  the  standard 
84%)  maybe  this  would  be  a  tool  for  decision  making. 

Visibility  by  WS  is  valuable  for  those  items  which  base 
supply  is  authorized  stockage.  Although  I'm  not  sure  what, 
if  any,  effect  a  retail  supply  account  has  on  this, 
especially  if  not  hand  massaging  the  process.  Does  it 
really  indicate  a  continuous  improvement  measure  to  control 
the  process  -  NO! 

I  think  this  is  a  good  indicator  when  viewed  from  the 
customer's  point  of  view.  He  doesn't  really  care  whether  or 
not  it's  authorized  for  stock.  If  he  needs  it,  he  needs  it! 
We  tend  to  hide  behind  the  "I'm  not  authorized  to  stock 
that"  syndrome. 

I  think  this  is  one  which  would  be  interesting  to  test  in 
the  field.  With  all  the  "complaints"  in  the  earlier  session 
about  the  usefulness  or  "un-usefulness"  of  issue 
effectiveness  figures,  the  fact  that  this  one  is  by  WS,  and 
also  does  not  include  authorized  figures  makes  it 
intriguing.  I  think  it's  worth  a  shot.  It  could  be  dropped 
or  altered  later  if  found  to  be  lacking. 

This  is  a  standard  that  Supply  has  dealt  with  for  ever. 

With  limited  funding  in  the  future  and  so  many  other 
organizations  with  the  capability  to  bypass  supply,  this  may 
not  be  a  true  way  to  evaluate  supply.  Keep  it  as  a 
management  tool . 


5.  NOT  MISSION  CAPABLE  SUPPLY  RATE 


Great,  you  removed  NMCB  from  the  formula.  Now,  if  we  could 
differentiate  transportation  vs  supply  this  indicator  would 
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be  awesome.  As  strides  are  made  in  providing  full  pipeline 
visibility  (a  goal  in  the  two  levels  of  maintenance 
project!)  this  differentiation  may  become  a  reality. 

I  think  that  this  will  be  useful  information  to  supply 
managers  on  how  well  MI CAP  is  supporting  the  customer. 

Agree  with  other  comment  about  segregation  of  other 
responsible  activities/agencies  for  greater  visibility  since 
they  are  "suppliers"  to  our  process  of  providing  parts  to 
our  customer.  Perhaps  then,  we  could  tighten  the  control 
limits  and  achieve  REAL  results. 

Don't  think  this  is  useful.  We  used  it  for  years  and  blamed 
maintenance  for  time  they  were  working  on  the  airplane,  even 
though  we  still  need  the  part.  Doesn't  drive  any  behavior 
to  improve  support.  It  does  drive  behavior  to  try  to  code 
time  to  maintenance  and  that  doesn't  get  the  part  any 
faster.  When  we  changed  to  TNMCS  years  ago  it  helped  us  take 
responsibility  for  ALL  the  time  for  which  we  owed 
maintenance  a  part.  I  think  that  was  healthy. 

I  still  think  this  is  good  for  the  COS  ONLY.  Unfortunately, 
it  drives  wings  to  "blame"  different  organizations  for 
delays,  etc.  This  is  an  indicator  of  "how  things  went,"  not 
a  tool  for  "making  things  better." 

This  indicator  is  a  blame  them  not  me  generator.  Creates 
friction  between  Micap  and  Maintenance  over  what  should  be 
ordered  Micap  and  what  shouldn't.  Still  a  good  indicator 
for  supply  to  show  it  really  does  play  a  big  part  in  the 
wing's  mission. 


6.  TOTAL  NOT  MISSION  CAPABLE  SUPPLY 


Again,  this  is  as  good  as  it  gets  until  total  reparable 
pipeline  visibility  becomes  a  reality.  Supply  can  make 
incremental  improvements  in  this  arena. 

This  didn't  change,  but  I  see  it  as  a  measure  of  cooperation 
between  supply  and  its  customer,  maintenance.  If  we  ever 
get  to  measuring  prevention  points  in  these  processes,  then 
we  can  strive  to  improve  end  products.  As  it  currently  is, 
this  is  an  end  of  the  process  measure  that  detects,  not 
prevents  anything. 

See  comments  on  NMCS 

This  is  a  better  indicator  for  use  by  the  senior  managers  at 
a  wing.  The  main  thing  lacking  is  something  that  can  tell 
how  much  time  is  being  consumed  due  to  transportation  or 
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funding  problems.  Again,  I'm  not  too  sure  how  much  this 
number  is  going  to  help  us  make  things  better. 

If  your  managing  the  NMCS  rate  you  really  don't  need  to 
worry  about  this  one  too 


7.  RECEIPT— NOT-DUE— IN  RATE 


I  like  this  computed  as  a  rate.  The  rate  figure  gives  the 
supply  manager  a  better  indication  of  the  relative  size  of 
this  problem  indicator. 

As  a  rate,  th  lets  supply  know  the  magnitude  of  potential 
problems.  Just  numbers  never  did  that.  Combined  with  other 
data,  such  as  where  did  the  RNDIs  in  come  from,  meaningful 
analysis  can  be  done.  This  can  affect  several  areas  of  the 
account . 

Agree  with  other  comments.  A  rate  is  much  more  effective. 

It  normalizes,  for  example,  those  months  which  have  high 
activity . 

Not  to  necessarily  disagree  with  the  others,  I  believe  this 
rate  will  become  more  and  more  meaningLESS  as  time  goes  by. 
DRIVE  and  other  new  techniques  for  managing  stock  in  the 
future  (such  as  Lean  Logistics/ Just-In-Time,  or  whatever 
else  you  want  to  call  it)  are  going  to  result  in  parts  being 
pushed  to  the  bases  instead  of  us  ordering  them  as  we  have 
in  the  past.  This  will  result  in  an  astronomical  jump  in 
the  RNDI  rate,  but  it  will  be  good,  not  bad.  These  new 
methods  of  managing  the  AF  inventory  will  make  this  measure 
obsolete  and  useless. 

Supply  is  the  recipient  of  this  indicator  with  limited 
control  on  incoming  property .  Depot  sends  it  and  base  supply 
receives  it.  Very  little  action  is  ever  taken  to  do 
anything  with  this 


8.  PERCENT  REPAIRABLE  THIS  STATION  (3-LEVEL  MX  ONLY) 


A  matrix  that  the  LG  needs  to  be  aware  of.  This  is  not 
necessarily  a  sound  indicator  for  supply  management 
improvement,  but  we  have  the  capability  to  provide  this  data 
to  someone  who  can  better  effect  the  results. 

This  could  be  useful  data  for  MICAP  research,  stock  control 
for  adjusting  stock  levels,  bench  stock  adjustments  (items 
to  support  the  reparables),  etc.  Since  supply  has  the 


129 


capability  to  track,  could  be  provided  to  the  customer  for 
his/her  use. 

Agree  with  other  comments.  Does  data  system  let  us 
differentiate  between  to  and  three  level  items  now?  If  not, 
we  will  need  to  change  select  criteria  for  management 
reports . 

Although  this  is  not  a  meaningful  measure  of  Supply  activity 
or  problems,  it  does  give  us  a  useful  tool  by  which  we  can 
question  Maintenance  and  help  them  in  their  ability  to 
support  the  WS.  I  believe  the  MICAPers  view  this  ability  as 
helping  to  make  them  a  more  integral  part  of  the 
Supply /Maintenance  team. 

Interesting  management  indicator  but  not  a  true  supply 
indicator,  give  it  to  maintenance 


INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 


Inventory  accuracy  as  a  total  picture  (say  monthly)  does  not 
provide  an  indicator  for  improvement .  Because  of  the  sheer 
volume,  this  indicator  gets  washed.  To  provide  a  clear 
picture,  recommend  this  indicator  be  displayed  by  warehouse 
zones  or  sub— warehouses  so  that  specific  improvements  could 
be  targeted  where  required. 

I  have  to  agree  with  the  comment  about  stratifying  the  data 
by  warehouse,  AND  type  of  inventory  accomplished.  This 
would  be  more  useful  to  the  warehouse  supervisor,  inventory 
supervisor,  the  COS  and  managers  in  between.  I  suppose  a 
roll-up,  such  as  this,  is  necessary  for  a  quick  snapshot, 
but  it  doesn't  help  to  pinpoint  areas  for  improvement  in 
procedures,  training,  security,  etc.  that  more  definitive 
data  would. 

Agree  with  foregoing  comments.  But  what  about  an  indicator 
called  "initial  inventory  accuracy".  The  ALCs  use  that 
ratio  to  describe  how  accurate  the  inventory  is  upon  initial 
request  by  the  customer.  It  is  computed  using  a  500  item 
sample  once  a  quarter  and  measures  items  over  or  short  as  a 
ratio  to  total  items  BEFORE  recounts  and  transaction 
research  is  done.  Rationale  for  this  is  that  the  customer 
doesn't  want  to  wait  while  we  fix  the  record.  If  it's 
accurate  from  the  start,  he  gets  his  part  immediately.  In 
the  SBSS  we  might  measure  this  as  we  go  along  in  the  annual 
complete  inventory  schedule  rather  than  a  sample. 

Agree  with  all  comments  above.  However,  I  believe  that  the 
"initial  inventory  accuracy"  rate  be  computed  AFTER  the 
initial  recount.  i.e.  the  inventory  person  makes  the  count 
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and  the  initial  check  with  the  computer  shows  the  count  to 
be  "over  by  2  items".  The  inventory  person  then  goes  back 
and  does  a  recount  and  discovers  that  a  mistake  in  counting 
had  been  made  during  the  initial  count.  This  should  not 
count  against  the  reported  accuracy  rate.  What  we're 
looking  for  should  be  an  accurate  view  of  where  we  stand 
BEFORE  automatic  adjustments  are  made,  but  AFTER  "dumb-shit" 
errors  are  discounted. 

I  use  this  indicator  only  by  warehouse,  overall  total 
doesn't  mean  much 


10.  SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 


While  $  value  gives  a  more  meaningful  measure  than  just 
numbers  of  I/Rs,  if  stratifiable  by  weapons  system,  this 
could  be  really  useful. 

This  is  such  rare  event  that  it  should  be  indicated  by  the 
actual  number  of  serviceable  balances  with  no  location. 

When  provided  to  the  LGSD  Supervisor,  it  can  pinpoint 
training  deficiencies  or  simply  reaffirm  the  rare  event, 
(statistically  acceptable!) 

Agree  with  other  comments 

I  personally  don't  care  for  the  use  of  $  value  here.  The 
low  number  of  problems  in  this  area  offer  the  capability  for 
the  supply  manager  to  find  out  if  there  might  be  problems  in 
the  warehousing  process,  but  as  I  stated  earlier,  if  $ 
values  are  the  criteria,  then  one  high  dollar  value  item 
might  equal  100  low  dollar  items.  One  mistake  might  not 
indicate  a  problem,  whereas  100  probably  would.  If  the 
dollar  value  of  the  one  item  equals  the  dollar  value  of  the 
100  items,  we  might  never  realize  that  a  problem  existed! 

Not  a  big  issue,  but  don't  have  any  thing  better  to 
recommend 


11.  ITEM  RECORDS  FROZEN  FOR  INVENTORY  RATE 


I  like  this  as  a  measure  of  items  that  are  undergoing 
special  inventory.  Goal  in  any  supply  organization  should 
be  to  keep  specials  down  to  a  minimum.  By  making  it  a 
"ratio"  you've  leveled  the  playing  field  between  large  and 
small  organizations. 

I  like  this  expressed  as  a  ratio! 
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Ratio  much  better  than  numbers  for  trend  analysis  and 
comparisons,  but  knowledge  of  the  individual  incidents  is 
also  required  for  first  level  supervisor  action. 

Go  for  it. 

Disagree  with  some  comments.  I  would  rather  see  the  numbers 
rather  than  a  ratio.  The  ratio  should  always  be  99  percent, 
numbers  tell  you  when,  where,  and  why  problems  exist 


12.  TIME  TO  CLEAR  FROZEN  ITEM  RECORDS 


I  like  this,  too.  It  goes  hand-in-glove  with  #11.  If  the 
measure  self-stratifies  (ages)  into  days  (1,2, 3, 4, >5)  would 
also  be  valuable  to  managers  in  determining  responsiveness 
to  special  occurrences  within  the  account  (i.e.  a  possible 
measure  of  customer  responsiveness) .  If  once  programmed  to 
be  collected,  then  managers  could  get  a  special  run  by  W/S 
if  desired. 

This  indicator  provides  good  efficiency  data.  How  rapid  do 
my  inventory  folks  resolve  discrepancies?  I  would  not  want 
many  items  to  sit  unattended  very  long. 

Agree  with  other  comments.  This  is  a  useful  addition  to  the 
family  of  metrics  for  supply.  We  always  watched  this  at 
accounts  where  I  was  assigned. 

This  one  looks  like  it's  going  to  be  a  real  winner!  It 
might  be  useful  to  somehow  be  able  to  identify  these  items 
by  warehouse  number  so  that  it  would  be  easier  to  identify 
where  problem  areas  may  be  developing. 

Good  indicator.  Keep  this  one 


13.  REVERSE  POST  RATE 


As  always,  the  goal  is  zero.  But  this  provides  an  order  of 
magnitude  based  on  account  activity  in  lieu  of  just  the 
number  of  reverse  posts  that  took  place. 

Expressed  as  a  rate,  this  indicator  is  now  improved. 

Much  better  as  a  rate. 

I  like  it  better  as  a  rate,  but  I  would  also  like  to  be  able 
to  have  the  ability  to  distinguish  the  rate  by  transaction 
type.  I'm  sure  this  could  be  developed  at  the  base,  but  I 
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just  think  it  would  be  good  to  better  identify  where  the 
problems  are  really  located. 


Concur  with  other  comments/  this  can  open  a  bucket  of  worms 
when  used  for  analysis  purposes 


14.  WAREHOUSE  REFUSAL  RATE 


Here  again,  goal  is  zero,  but  the  order  of  magnitude  is  a 
better  indicator.  Often  took  the  numbers  and  computed  a 
rate  before  -  perspective  was  much  clearer. 

Rate  computation  gives  manager  a  basis  of  magnitude.  Great! 

Better  as  a  rate.  Suggest  stratification  by  warehouse  to 
pinpoint  where  the  problems  are. 

Good.  Now  we  need  to  see  WHICH  warehouses  are  having  the 
problems.  Same  comments,  basically,  as  for  Reverse  Post 
Rate . 

This  figure/rate  should  be  small  but  should  be  tracked  to 
keep  the  warehouse  on  a  positive  role 


15.  BENCH  STOCK  AVAILABILITY  (BY  ORGANIZATION) 


By  organization,  surely  gives  a  much  clearer  picture  of 
support  to  the  individual  customer.  In  this  day  of 
"customer  satisfaction"  this  gives  us  a  singular 
point-in-time  picture.  What  it  doesn't  convey  are  those 
items  that  for  funding  constraints  haven't  been  ordered; 
those  items  which  are  first  time  adds  to  the  customer's 
bench  stock.  These  special  conditions  are  known  only 
through  separate  knowledge  and  documentation. 

Expressed  by  organization,  this  indicator  provides  for 
continuous  and  specific  customer  service  gains. 

Agree  with  foregoing. 

Agree  with  above,  but  I  believe  in  most  cases,  the  COS  is 
not  going  to  have  many  items  not  ordered  due  to  lack  of 
funds  since  most  of  the  bench  stock  stuff  are  low  dollar 
items . 

Good  indicator  by  organization 


16.  NOT-STOCKED  BACKORDER  RATIO 


With  this  I  can  segregate  those  items  over  which  I  have  some 
control  from  those  which  I  have  absolutely  zero  control 
(4Ws) .  For  new  weapons  systems  being  brought  on-line  this 
would  be  extremely  meaningful.  I  would  think  that  the  Chief 
of  Maintenance  would  also  find  this  useful  information. 

Marry  this  indicator  with  the  stockage  effectiveness 
indicator.  Shown  side-by-side  this  provides  specific 
insight  you  can  share  with  your  customers  in  pursuit  of 
closer  customer  relations. 

Agree  with  foregoing  comments. 

I  don't  have  any  problems  with  this  on  paper.  I  would  love 
to  see  it  in  use  to  determine  if  it  needs  to  be  tweaked  some 
more . 

I  like  this  one.  Hopefully  it  will  show  the  customer  that 
they  are  buying  things  Supply  doesn't  stock 


17.  BENCH  STOCK-MICAP  RATE 


YES!  YES!  YES!  Too  many  times  close  to  fiscal  year  end,  we 
cut  back  on  bench  stock  fill  to  control  expenditures.  This 
would  give  all  concerned  a  measure  that  could  be  used  to 
beef  up  stock  on  certain  NSNs,  to  preclude  the  costly 
MICAPs.  For  want  of  a  bench  stock  item,  the  mission  wasn't 
flown  -  what  a  sad  and  unnecessary  thing  to  occur. 

Good  Stuff!  Why  let  small  dollar  value  items  ground  systems 
or  aircraft.  The  LG  might  want  to  talk  to  the  maintenance 
bench  stock  folks (for  failing  to  fill  B/S)  or  the  supply 
folks (for  not  having  the  restock  assets  when  required. 

Good  way  to  zero  in  on  problems  caused  by  small  items.  I 
think  most  of  these  have  been  lead  time  or  procurement 
problems  in  the  past,  but  with  SSD  obligation  authority 
restrictions  we' re  now  experiencing,  the  funding  issue  will 
become  the  most  important  in  the  future. 

I  REALLY  LIKE  THIS  ONE!  Should  be  very  useful  in  showing 
how  the  bench  stock  items  are  affecting  mission  support. 

Not  sure  this  will  prove  to  be  useful,  but  it  should  be 
implemented  on  a  test  basis  to  see  what  develops 
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18.  SUPPLY  CANN  RATE  (BY  WEAPON  SYSTEM) 


In  an  effort  to  improve  customer  service,  this  indicator 
says  a  lot.  How  much  work  is  supply  forcing  on  maint.  for 
lack  of  asset?  Is  the  item  critically  short  everywhere? 
(Item  manager,  where  are  you?)  (HQ,  What  are  you  doing?) 

Don't  think  that  this  is  available  in  supply's  database. 
Would  be  obtained  from  Maintenance,  right?  Not  sure  that 
base  supply  can  have  significant  impact  to  improve  the 
stockage  effectiveness  to  lower  this  rate.  However,  it 
would  provide  data  for  supply  to  use  in  "negotiations”  with 
Item  Managers  and  HQs  staffs  that  monkey  around  with  the 
UMMIPS  priority  system  to  achieve  hidden  agendas  (good  rates 
for  one  base  vs  another;  level  #  down  aircraft  among  bases; 
retain  stocks  for  really  high  priorities,  etc.) 

Not  new.  Most  accounts  have  tracked  this  for  years.  It  is 
a  good  indicator  of  how  much  work  the  supply  system  is 
causing  maintenance.  What  behavior  does  it  drive?  Does  it 
make  supply  work  harder  to  get  parts  on  the  shelf?  Takes 
some  leadership  to  make  the  connection  to  the  troops.  This 
is  a  powerful  metric  in  the  hands  of  perceptive  leadership. 

Very  good  comments  above.  This  measure  is  better  used 
locally  by  knowledgeable  managers  as  opposed  to  command-wide 
comparisons.  The  reason  for  this  is  that  there  are 
differing  views  on  how  and  when  canns  should  and  will  be 
used  at  different  wings.  All  elements  being  equal,  some 
wings  will  cann  a  part  while  others  will  not  —  all  for  good 
but  different  reasons.  Thus,  a  higher  cann  rate  for  one 
wing  may  not  mean  that  Supply  is  doing  a  worse  job  in 
supporting  maintenance  than  another  with  a  lower  cann  rate. 
Taken  in-house,  though,  and  measured  over  time,  this  measure 
can  show  meaningful  trends. 

I  like  this  indicator  because  it  gets  Supply  and  Maintenance 
talking  together  to  solve  a  common  problem 


I 

4 


135 


Appendix  F:  Ratings  From  Second  Focus  Group 


»  Equally  Weighted  Criteria  (Text  Report)  « 


Median 

Mean 

18 

17.2 

18 

16.6 

16 

15.8 

16 

15.2 

15 

14.4 

15 

14.0 

15 

13.8 

15 

13.6 

14 

14.4 

14 

13.2 

14 

12.6 

12 

12.2 

12 

11.8 

12 

10.6 

11 

12.0 

11 

10.8 

9 

9.8 

9 

3.2 

**  ALTERNATIVES  ** 


BENCH  STOCK-MICAP  RATE 

BENCH  STOCK  AVAILABILITY  (BY  ORGANIZATION) 

TIME  TO  CLEAR  FROZEN  ITEM  RECORDS 
SUPPLY  CANN  RATE  (BY  WEAPON  SYSTEM) 

STOCKAGE  EFFECTIVENESS  (EACH  WEAPON  SYSTEM) 

ISSUE  EFFECTIVENESS  -  (EACH  WEAPON  SYSTEM) 

NOT  MISSION  CAPABLE  SUPPLY  RATE 
WAREHOUSE  REFUSAL  RATE 

ITEM  RECORDS  FROZEN  FOR  INVENTORY  RATE 

REVERSE  POST  RATE 

TOTAL  NOT  MISSION  CAPABLE  SUPPLY 

NOT -STOCKED  BACKORDER  RATIO 

INVENTORY  EFFECTIVENESS  RATIO  -  (OVERALL) 

UNREPORTED  EXCESS  INVENTORY  RATIO 

PERCENT  REPAIRABLE  THIS  STATION  (3-LEVEL  MX  ONLY) 
INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 
RECEIPT-NOT-DUE-IN  RATE 

SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 


»  Criteria  Matrix  « 

1.  How  well  does  this  indicator  reflect  customer/mission  support? 

Participant  Ratings 


**  Alternative  ** 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

N 

n 

MN 

STD 

MED 

UNREPORTED  EXCESS  IN 

- 

- 

2 

- 

1 

2 

- 

- 

- 

- 

5 

5 

4.6 

1.53 

5 

STOCKAGE  EFFECTIVENE 

- 

- 

- 

- 

- 

- 

1 

4 

- 

- 

5 

5 

7.8 

0.40 

8 

INVENTORY  EFFECT I VEN 

- 

- 

- 

1 

1 

2 

“ 

- 

1 

- 

5 

5 

6.0 

1.67 

6 

ISSUE  EFFECTIVENESS 

- 

- 

- 

- 

1 

- 

1 

1 

2 

- 

5 

5 

7.6 

1.50 

8 

NOT  MISSION  CAPABLE 

- 

- 

1 

- 

1 

- 

1 

1 

1 

- 

5 

5 

7.0 

1.67 

7 

TOTAL  NOT  MISSION  CA 

- 

- 

- 

- 

2 

- 

z 

- 

1 

5 

5 

6.6 

1.50 

7 

RECEIPT-NOT-DUE-IN  R 

- 

1 

1 

1 

1 

1 

- 

- 

- 

5 

5 

4.2 

1.72 

4 

PERCENT  REPAIRABLE  T 

- 

1 

- 

- 

2 

- 

1 

1 

- 

- 

5 

5 

5.4 

2.06 

5 

INVENTORY  ACCURACY  ( 

- 

1 

- 

2 

- 

1 

1 

- 

- 

- 

5 

5 

4.6 

1.74 

4 

SERVICEABLE  BALANCE 

- 

2 

1 

- 

1 

1 

- 

- 

- 

5 

5 

3.6 

1.62 

3 

ITEM  RECORDS  FROZEN 

- 

- 

- 

- 

1 

2 

1 

1 

- 

- 

5 

5 

6.4 

1.02 

6 

TIME  TO  CLEAR  FROZEN 

- 

- 

- 

- 

- 

- 

3 

1 

1 

- 

5 

5 

7.6 

0.80 

7 

REVERSE  POST  RATE 

- 

- 

- 

- 

2 

2 

1 

- 

- 

- 

5 

5 

5.8 

0.75 

6 

WAREHOUSE  REFUSAL  RA 

- 

- 

- 

1 

- 

- 

2 

1 

1 

- 

5 

5 

7.0 

1.67 

7 

BENCH  STOCK  AVAILABL 

- 

- 

- 

- 

- 

1 

1 

3 

- 

5 

5 

8.4 

0.80 

9 

NOT-STOCKED  BACKORDE 

- 

- 

- 

- 

1 

3 

1 

- 

- 

- 

5 

5 

6.0 

0.63 

6 

BENCH  STOCK-MICAP  RA 

- 

- 

- 

- 

- 

1 

- 

- 

2 

2 

5 

5 

8.8 

1.47 

9 

SUPPLY  CANN  RATE  (BY 

- 

- 

- 

- 

- 

1 

1 

2 

1 

- 

5 

5 

7.6 

1.02 

8 
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2.  How  well  does  this  indicator  lead  you  to  continually  improve  the 
process 

Participant  Ratings 

**  Alternative  **  123456789  10  NnMN  STD  MED 


UNREPORTED  EXCESS  IN  - 
STOCKAGE  EFFECTIVENE  - 
INVENTORY  EFFECT I VEN  - 
ISSUE  EFFECTIVENESS 
NOT  MISSION  CAPABLE 
TOTAL  NOT  MISSION  CA  - 
RECEIPT-NOT-DUE-IN  R  - 
PERCENT  REPAIRABLE  T  - 
INVENTORY  ACCURACY  (  1 

SERVICEABLE  BALANCE 
ITEM  RECORDS  FROZEN 
TIME  TO  CLEAR  FROZEN  - 
REVERSE  POST  RATE 
WAREHOUSE  REFUSAL  RA  - 
BENCH  STOCK  AVAILABL  - 
NOT-STOCKED  BACKORDE  - 
BENCH  STOCK-MICAP  RA  - 
SUPPLY  CANN  RATE  (BY  - 


-  -  1  1  -  2  1 

---1121 
--1-31- 
1  -  -  -  -  2  2 

-  -  1  -  1  -  3 

1--1-2- 

-  1  1  1  -  -  2 

-  --121- 

1  -  -  1  1  1  1 

-  -  -  -  1  -  3 

-  -  -  1  -  -  1 

-  -  -  1  -  1  2 

---11-2 
-  -  -  -  1  -  1 

-  -  1  -  2  1  1 

-----11 
-  -  -  -  1  -  4 


-  -  5  5  6.0  1.41  7 

-  -  5  5  6.6  1.02  7 

-  -  5  5  5.8  0.98  6 

-  -  5  5  6.4  2.24  7 

-  -  5  5  6.8  1.60  8 

1  -  5  5  6.0  2.37  7 

-  -  5  5  5.6  2.06  5 

1  -  5  5  6.6  1.36  6 

1  -  5  5  6.2  2.71  7 

-  -  5  5  5.6  2.06  6 

-  1  5  5  8.0  1.26  8 

2  1  5  5  8.2  1.72  9 

1  -  5  5  7.4  1.36  8 

1  -  5  5  7.2  1.47  8 

3  -  5  5  8.2  1.16  9 

-  -  5  5  6.2  1.32  6 

3  -  5  5  8.4  0.80  9 

--55  7.6  0.80  8 
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INSTRUCTIONS 


The  purpose  of  this  research  is  to  try  to  develop 
supply  management  indicators  that  are  more  in  line  with  the 
TQM  philosophy.  Specifically,  to  find  useful  measures 
(metrics)  that  display  the  TQM  concepts  of  customer  focus 
and  continuous  process  improvement. 

The  attached  18  indicators  were  developed  through 
extensive  research  on  measurement  and  TQM.  They  are 
indicators  which  the  research  implies  are  better  management 
tools  than  many  currently  used  supply  indicators  and  can  be 
used  to  compare  dissimilar  operations. 

Please  rate  the  following  indicators  on  a  scale  of  1 
(low)  to  10  (high)  on  two  criteria. 

The  first  criteria  is: 

How  well  does  this  indicator  reflect  customer/mission 
support? 

The  second  criteria  is: 

How  well  will  this  indicator  lead  you  to  continually  improve 
the  process? 

Each  indicator  should  be  rated  on  its  own  merit,  not 
relative  to  the  other  indicators.  In  other  words,  it  is 
possible  that  they  can  all  be  tens  or  they  can  all  be  twos. 
Also,  just  because  an  indicator  rates  high  on  one  criteria 
does  not  necessarily  mean  that  it  will  rate  as  high  on  the 
other. 

Feel  free  to  make  any  comments  concerning  these 
indicators  in  the  space  provided.  Please  return  the 
completed  appraisal  in  the  enclosed  envelope. 

Thank  you  for  your  assistance. 


Brian  B.  Yoo,  Capt,  USAF  Michael  O.  Cannon,  lLt,  USAF 
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#1  -  UNREPORTED  EXCESS  INVENTORY  RATIO 


$  Value  of  Unreported  Excess  Material 
Total  $  Value  of  Inventory 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS :  _ _  _ 


#2  -  STOCKAGE  EFFECTIVENESS  -  (EACH  WEAPON  SYSTEM) 

_ No.  WS  Units  Issued _ 

No.  WS  Units  Issued  +  No.  WS  Units  B/O  -  No.  WS  Units  B/O  4 W 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


#3  -  INVENTORY  EFFECTIVENESS  RATIO  -  (OVERALL) 


Stockage  Effectiveness  x 


$  Value  of  On- hand  Inventory 
$  Value  of  Authorized  Inventory 


Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


139 


#4  -  ISSUE  EFFECTIVENESS  -  (EACH  WEAPON  SYSTEM) 

_ No.  NS  Units  Issued _ 

No.  WS  Units  issued  +  No.  WS  Units  B/O 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS:  _ _ _ 


#5  -  NOT  MISSION  CAPABLE  SUPPLY  RATE 

_ NMCS  Hours _ 

Total  Possessed  Hours 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS: 


#6  -  TOTAL  NOT  MISSION  CAPABLE  SUPPLY 

NMCS  +  NMCB  Hours 
Total  Possessed  Hours 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 


COMMENTS : 


#7  -  RECEIPT-NOT-DUE-IN  RATE 

Total  Shipments  Received  With  No  Corresponding  Due- in 
Total  Number  of  Receipts 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS : _ _ 


#8  -  PERCENT  REPAIRABLE  THIS  STATION  (3-LEVEL  MX  ONLY) 

Number  3  Level  Maintenance  Units  Repairable  This  Station 
No.  3LM  Units  RTS  -  No.  3 LM  Units  NRTS  -  No.  3 LM  Units  Cond 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


#9  -  INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 

1-  rofcai  Units  Over  +  Short 
Total  Record  Balance 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS:  _ _ 
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#10  -  SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 

$  Value  of  Item  Records  With  Serv  Bal  and  No  Whse  Location 
Total  $  Value  of  inventory 

Customer/Mission  Focus  12345678910 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


#11  -  ITEM  RECORDS  FROZEN  FOR  INVENTORY  RATE 

Item  Records  Frozen  for  Non-routine  Inventory 
Total  Item  Records 


Customer/Mission  Focus  12345678910 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


#12  -  TIME  TO  CLEAR  FROZEN  ITEM  RECORDS 

Average  Time  to  Clear  Non-routine  Frozen  Item  Records 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  123456789  10 


COMMENTS : 


#13  -  REVERSE  POST  RATE 

Number  of  Reverse  Post  Transactions 
Total  Number  of  Transactions 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


#14  -  WAREHOUSE  REFUSAL  RATE 

Total  Number  of  Warehouse  Refusals 
Total  Number  of  Issues 

Customer/Mission  Focus  12345678910 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


#15  -  BENCH  STOCK  AVAILABILITY  (BY  ORGANIZATION) 

Total  Line  Items  Authorized  -  Line  Items  Due-out 
Total  Line  Items  Authorized 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 


COMMENTS : 


1 


#16  -  MOT-STOCKED  BACKORDER  RATIO 

Number  of  Units  Backordered  4 W 
Total  Number  of  Units  Backordered 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


#17  -  BENCH  STOCK-MI CAP  RATE 

Number  of  MI  CAP  Hours  Due  to  Bench  Stocked  Items 
Total  Number  of  MICAP  Hours 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


#18  -  SUPPLY  CANN  RATE 

Number  of  CANN  Actions  to  Clear  +  Prevent  MICAPs 
Total  Number  of  CANN  Actions 

Customer/Mission  Focus  123456789  10 
Continuous  Improvement  12345678910 

COMMENTS:  _ 


» 
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Appendix  H:  Ratings  From  Questionnaire 


»  Equally  Weighted  Criteria  (Text  Report)  « 


Median 

Mean 

19 

17.5 

18 

17.3 

18 

17.1 

18 

16.8 

17 

16.9 

17 

16.0 

17 

15.9 

16 

15.9 

16 

15.7 

16 

15.6 

16 

14.3 

15 

14.2 

14.5 

14.1 

14 

13.9 

14 

12.7 

14 

12.2 

13 

12.7 

12 

11.9 

**  ALTERNATIVES  ** 


NOT  MISSION  CAPABLE  SUPPLY  RATE 

BENCH  STOCK  AVAILABILITY  (BY  ORGANIZATION) 

STOCKAGE  EFFECTIVENESS  (EACH  WEAPON  SYSTEM) 

BENCH  STOCK-MICAP  RATE 

TOTAL  NOT  MISSION  CAPABLE  SUPPLY 

INVENTORY  ACCURACY  (COMPLETED  PLUS  SPECIAL) 

SERVICEABLE  BALANCE  WITH  NO  WAREHOUSE  LOCATION 

WAREHOUSE  REFUSAL  RATE 

SUPPLY  CANN  RATE  (BY  WEAPON  SYSTEM) 

ISSUE  EFFECTIVENESS  -  (EACH  WEAPON  SYSTEM) 

PERCENT  REPAIRABLE  THIS  STATION  (3-LEVEL  MX  ONLY) 

REVERSE  POST  RATE 

TIME  TO  CLEAR  FROZEN  ITEM  RECORDS 

RECEIPT-NOT-DUE-IN  RATE 

UNREPORTED  EXCESS  INVENTORY  RATIO 

NOT-STOCKED  BACKORDER  RATIO 

ITEM  RECORDS  FROZEN  FOR  INVENTORY  RATE 

INVENTORY  EFFECTIVENESS  RATIO  -  (OVERALL) 


»  Criteria  Matrix  « 

1.  How  well  does  this  indicator  reflect  customer/mission  support? 


Participant  Ratings 


**  Alternative  ** 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

N 

n 

MN 

STD 

MED 

UNREPORTED  EXCESS  IN 

4 

1 

2 

2 

1 

2 

4 

3 

- 

2 

21 

21 

5.2 

2.98 

6 

STOCKAGE  EFFECTIVENE 

- 

- 

1 

- 

- 

1 

2 

3 

7 

7 

21 

21 

8.6 

1.72 

9 

INVENTORY  EFFECT IVEN 

2 

- 

4 

3 

1 

- 

4 

- 

3 

2 

21 

19 

5.6 

2.97 

5 

ISSUE  EFFECTIVENESS 

- 

1 

- 

1 

- 

3 

1 

5 

3 

7 

21 

21 

8.0 

2 . 19 

8 

NOT  MISSION  CAPABLE 

- 

- 

- 

- 

1 

1 

2 

2 

4 

11 

21 

21 

8.9 

1.51 

10 

TOTAL  NOT  MISSION  CA 

- 

- 

- 

- 

1 

2 

2 

2 

5 

8 

21 

20 

8.6 

1.60 

9 

RECEIPT-NOT-DUE-IN  R 

1 

2 

2 

2 

2 

2 

5 

1 

3 

1 

21 

21 

5.8 

2.61 

6 

PERCENT  REPAIRABLE  T 

- 

1 

2 

- 

1 

2 

3 

5 

2 

5 

21 

21 

7.3 

2.44 

8 

INVENTORY  ACCURACY 

- 

2 

- 

1 

1 

1 

5 

2 

6 

3 

21 

21 

7.3 

2.39 

8 

SERVICEABLE  BALANCE 

- 

1 

1 

- 

3 

1 

3 

4 

6 

2 

21 

21 

7.3 

2.22 

8 

ITEM  RECORDS  FROZEN 

- 

1 

3 

2 

5 

3 

1 

3 

2 

1 

21 

21 

5.8 

2.26 

5 

TIME  TO  CLEAR  FROZEN 

- 

- 

1 

2 

4 

3 

2 

6 

1 

2 

21 

21 

6.7 

1.98 

7 

REVERSE  POST  RATE 

- 

- 

4 

1 

4 

1 

4 

3 

3 

1 

21 

21 

6.2 

2.26 

7 

WAREHOUSE  REFUSAL  RA 

- 

- 

- 

1 

- 

4 

3 

5 

4 

4 

21 

21 

7.9 

1.65 

8 

BENCH  STOCK  AVAILABL 

- 

- 

- 

- 

- 

1 

1 

4 

10 

'  5 

21 

21 

8.8 

1.03 

9 

NOT-STOCKED  BACKORDE 

1 

1 

- 

1 

3 

3 

6 

4 

1 

1 

21 

21 

6.3 

2.15 

7 

BENCH  STOCK-MICAP  RA 

- 

- 

1 

- 

1 

2 

2 

10 

5 

21 

21 

8.5 

1.66 

9 

SUPP  CANN  RATE  (BY 

- 

1 

1 

- 

- 

- 

5 

4 

5 

4 

21 

20 

7.9 

2.13 

8 
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2.  How  well  does  this  indicator  lead  you  to  continually  improve  the 
process 


Participant  Ratings 

**  Alternative  **  123456789  10  NnMN  STD  MED 


UNREPORTED  EXCESS  IN  -  -  1  -  1  3  5  5  3  3  21  21  7. 51. 75  8 

STOCKAGE  EFFECTIVENE  -  -  -  -  1  1  2  4  6  7  21  21  8. 61. 43  9 

INVENTORY  EFFECTIVEN  2  -  2  2  1  -  5  1  3  32119  6. 42. 97  7 

ISSUE  EFFECTIVENESS  1  1  -  1  -  2  2  5  4  5  21  21  7. 62. 56  8 

NOT  MISSION  CAPABLE  -  -  -  -  1  -  3  6  3  82121  8. 61. 40  9 

TOTAL  NOT  MISSION  CA  -  -  -  -  1  2  2  6  3  6  21  20  8. 31. 53  8 

RECEIPT-NOT-DUE-IN  R  -  -  -  -  1  2  2  8  5  3  21  21  8. 11. 34  8 

PERCENT  REPAIRABLE  T  1  3  -  -  1  1  3  4  5  32121  7. 02. 89  8 

INVENTORY  ACCURACY  -  -  1  -  -  -  -  5  12  3  21  21  8. 61. 43  9 

SERVICEABLE  BALANCE  -  -  -  -  -  -  2  8  7  42121  8. 60. 92  9 

ITEM  RECORDS  FROZEN  -  1  1  1  1  4  1  8  3  12121  7. 02. 09  8 

TIME  TO  CLEAR  FROZEN  -  -  1  -  1  5  1  8  2  3  21  21  7. 51. 78  8 

REVERSE  POST  RATE  -  -  -  1  2  -  3  5  7  3  21  21  8. 01. 67  8 

WAREHOUSE  REFUSAL  RA  -  -  -  1  -  1  6  5  2  6  21  21  8. 11. 61  8 

BENCH  STOCK  AVAILABL  -  -  -  1  -  1  1  7  4  7  21  21  8. 51. 54  9 

NOT-STOCKED  BACKORDE  2  1  1  -  3  3  6  4  1  -21  21  5. 92. 33  7 

BENCH  STOCK-MICAP  RA  -  -  1  -  2  -  2  2  8  6  21  21  8. 31. 93  9 

SUPPLY  CANN  RATE  (BY  -  -  1  -  1  1  3  7  4  3  21  20  7. 91. 73  8 


* 
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